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BACKGROUND TO THE STUDY 



Effective schooling 

The provision of schooling is one of the most massive and ubiquitous undertakings of 
the modem state. Schools exert a near uni^^ersal influence over the lives of young 
people for periods of 12 years or more. They are the chief means by which young 
people are educated and prepared for full and productive participation in society. 
Schools account for a significant proportion of public and private expenditure, as well 
as generating substantial paid employment. 

Not surprisingly, there has long been an interest in knowing how effective the 
provision of school education is and how it can be improved (e.g.. Chapman et al., 
1991; Coleman et ai, 1966; Cuttance, 1992; DES, 1984; Goodlad, 1982, 1983; 
McGaw et aL, 1992; NCEE, 1983; OECD, 1983, 1989; Reynolds & Cuttance, 1992; 
Rutter et ai, 1979). In country after country, the question has been raised as to 
whether schools are meeting society's expectations of them. This is an especially 
sensitive issue at the present time given the level of consensus regarding the 
importance of school education as an element of micro-economic reform and in 
meeting the demands of the modem workplace (e.g., Dawkins, 1988; OECD, 1986). 
Concern about the quality of school education has become a high priority policy issue 
in all OECD countries. Attention has focused on the operational definition and 
measurement of the quality of schools, the identification of factors associated with 
effective schooling, and the use of such knowledge to achieve further improvements in 
quality (see, for example, Mortimore, 1991). 

Early studies of school effectiveness such as those by Coleman et al. (1966) and Jencks 
et al. (1972) focused on the extent to which schools equalised opportunities for 
different groups within the community. The results seemed to point to the rather 
depressing conclusion that the influence of the school was insignificant compared with 
the influence of family background. Using different methodologies and asking different 
questions, more recent studies (e.g., Mortimer et al, 1988a, 1988b) have focused on 
identifying common characteristics of 'good' or 'effective' schools (i.e., schools that are 
evidently performing much better than others with which they might legitimately be 
compared taking into account their student intake) and have come to the conclusion 
that schools do make a difference. Moreover, there is now widespread agreement 
among those reviewing the research that effective schools typically exhibit: 

• strong educational leadership 

• high expectations of student achievement 

• an emphasis on basic skills 

• a safe and orderly climate 

• frequent evaluation of students' progress. 

While these findings make intuitive sense, for the following reasons it is necessary to 
treat them with caution. 
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1. The empirical evidence in support of the conclusions reached regarding the 
characteristics of effective schools is not extensive, with much of it derived from 
small-scale case studies (see Banks, 1992). For example, Rutter and colleagues' 
influential 1979 study. Fifteen thousand hours: Secondary schools and their 
effects on children, was based on observations made in just twelve inner London 
schools. Moreover, most studies have included only small numbers of outcome 
and explanatory variables. 

2. There have been few large-scale studies that have employed sample designs 
capable of providing valid generalisations about the characteristics of effective 
schools. Furthermore, until very recently, researchers have not had access to 
appropriate statistical tools with which to analyse the data from such studies. For 
example, access to multi-level analysis techniques has only been available in recent 
years (see Bock, 1989; Goldstein, 1987; Raudenbush & Willms, 1991). Only 
through the use of such techniques can proper account be taken of the 
organisational structure of schools, namely, the way in which whole classes of 
students relate to particular teachers, who in turn form the staff of a particular 
school (see Bryk & Raudenbush, 1989). Traditional regression analyses ignore this 
hierarchical clustering and commonly give rise to serious errors in estimating the 
influences operating at different levels. 

3. Most previous studies have been cross-sectional in nature and have not allowed for 
the measurement of growth and change. As a result, they have had to report 
estimates of school effectiveness based on average achievement levels statistically 
adjusted for relevant intake charcteristics. The problem with this approach is that 
it tends to lead to serious underadjustments. Only through a longitudinal study is it 
possible to estimate with confidence the specific effects of schools over and above 
what students bring with them. 

4. The identification of effective schools has been made on the basis of a wide range 
of operational definitions as to what it means to be a 'good' school. The most 
common method has been to choose those schools with scores on tests of reading 
and mathematics (or on public examinations), that are higher than average, after 
making adjustments for the nature of their student intakes. Studies such as those 
of Rutter et al. (1979) and Mortimore et al, (1988a, 1988b) have investigated 
multiple outcomes, but provide little insight into the relationships among various 
outcome and explanatory measures and the extent to which effective schools are 
consistently above average across all measures. A recent major survey (McGaw et 
al, 1992) has demonstrated that when the clients of schooling - parents, teachers 
and the community at large - are asked to describe what it is that makes for an 
effective school, they provide complex and multi-dimensional descriptions 
embracing many aspects of the total character of the school and the learning 
environment created by students, parents and teachers. 



The project 



The Victorian Quality Schools Project^ is a research and development project which 
aims to develop strategies for both schools and the system as a whole that will lead to 
substantially improved educational outcomes. The research component of the project 
was conceived in the light of the international literature on school effectiveness 
research and the perceived limitations of that research. In addition, it builds on the 
outcomes of an earlier, four-year longitudinal study of factors affecting students' 
reading achievement undertaken within the Victorian Ministry of Education, known as 
the "100 Schools Project - Literacy Program Study" (Rowe, 1990, 1991a, 1991b; 
Rowe & Rowe, 1992a, 1992b). 

The present study aims to provide answers to the following research questions: 

• What are the characteristics of schools in which students make rapid and 
sustained progress in English and Mathematics, after adjusting for their 
initial levels of achievement? 

• What are the characteristics of schools in which there are positive student 
attitudes and behaviours, positive perceptions by teachers of their work 
environment and high levels of parent participation in and satisfaction with 
their child*s schooling? 

In addition, the study aims to contribute developmentally towards other aims of the 
Victorian Quality Schools Project, namely: 

• to facilitate school improvement processes within participating schools, and 

• to provide input to system-level quality assurance and accountability 
arrangements, policy development and planning. 



Study design 

The overall design of the study is represented in schematic form in Figure 1 . The 
major feature of the design is the opportunity to explore inter-relationships among 
factors at three levels (student, teacher and school) and over three time periods (1992, 
1993 and 1994). This involves estimating the effects of factors at all three levels on a 
range of outcomes, as well as reciprocal relationships. An example of a reciprocal 
relationship would be the influence that schools and teachers have on students and in 
the opposite direction, the influence that students have on teachers and schools. 



Formerly known as the "150 Schools Project" 
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Figure L Schematic model of the study design 



In more detail, the study design incorporates several unique or unusual features, 
including: 

• a large sample of schools involving all students in each of five different Year levels, 
thus ensuring generalisability of findings and enabling class and teacher effects to 
be estimated properly 

• the use of multiple outcome measures, including students' achievements in English 
and mathematics, student attitudes and behaviours, teachers' perceptions of their 
work environment, and parent participation in and satisfaction with their child's 
school 

• the use of a longitudinal design to monitor students' progress and growth, as well 
as other changes within the school over time, and 

• the use of analytic methods that allow fijll exploration of interrelationships among 
factors at each level of analysis, as well as the simultaneous estimation of the 
effects of factors at the student, teacher and school levels on rates of student 
progress, adjusting for initial achievement levels. 



The sample 

The study uses a stratified probability sample of government and non-government 
primary schools and secondary colleges. The sample is designed to provide estimates 
which are accurate within 95% confidence limits. 



The sampling procedure involved a two-stage cluster design was used in which schools 
were selected with probability proportional to ^ize at the first stage, and entire year 
levels in the selected schools at the second stage. In 1992, data were collected from 
the entire cohort of students (and their parents and teachers) in year levels Prep, Year 
2, Year 4, Year 7 and Year 9. Repeated measures for those students remaining in the 
sampled schools will be obtained in 1993 and 1994, as follows: 



Year Levels 

1992 P 2 4 7 9 

1993 1 3 5 8 ^ 10 

1994 ^^2 ^^4 ^^9 ^11 



Following written invitations, a total of 96 schools agreed to participate in the study. 
Useable data were received from 90 school sites, representing a response rate of 88.5 
per cent. The number of schools that agreed to participate in the study was 
considerably smaller than anticipated due to a number of factors, including industrial 
action within government schools aimed at preventing the use of profiles in obtaining 
measures of achievement. On the other hand, the number of students for whom data 
were returned was higher than expected. The achieved sample in 1992 was 
distributed as shown in Tables 1.1 to 1.3 of Appendix 1. In summary, the achieved 
sample comprised 13,909 students and 931 teachers from 90 school sites. 



Procedure 

A set of four instruments was used in 1992: 

1 . Parent Questionnaire 

2. Student Record 

3. Teacher Record 

4. Teacher Questionnaire 

Following invitations to sampled schools to participate in the project, one teacher ft*om 
each selected school was invited to attend a professional development day to learn 
about the specific objectives, design and school-based administrative requirements of 
the project. Programs were run in each of the eight regions during August 1992. The 
selected teachers were then asked to train other teachers in the school where more 
than one class was involved. 
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Undertakings were given that absolute confidentiality of data sources would be strictly 
maintained and individual students, teachers or schools would not be identified, nor 
would inter-systemic comparisons be made. It was also agreed that individual school- 
and student-level data would be returned to the school and would remain its property. 



Measurement of variables 

Tables 2.1 and 2.2 in Appendix 2 summarise characteristics of the measures used in 
the study. Related technical details are provided in Appendix 5. 

Student home background characteristics 

The Year Level, Age, Gender, postcode of the student's residential address, whether 
the student was an Aboriginal/Torres Strait Islander, and whether the student was 
enrolled under the Integration Program, were recorded for each student. Students 
residing in areas/towns with a population of <25,000 were identified as Rural. A 
variable called Socio-Educational Level was measured for each student's family. This 
variable was a weighted composite comprising: the higher of mother's or father's 
occupational status using the Australian Bureau of Statistics classification (Castles, 
1986); the higher of mother's or father's number of years of education, and whether the 
student was in receipt of the Educational Maintenance Allowance (as a broad indir^^tor 
of family income). Non-English Speaking Background was measured as a weighted 
composite of each student's mother's and father's country of birth, and language(s) 
spoken at home. 



Student achievement 

The English and Mathematics achievements of students were recorded by teachers 
using the English Profiles (Victoria, 1991) and the Mathematics Profiles (Victoria, 
1992). Both the English and Mathematics Profiles are inventories consisting of 
multiple indicators describing observable learning behaviours that have been 
empirically calibrated on a common measurement scale and mapped onto a linear 
developmental growth continuum called "bands" (English) or "levels" (Mathematics) 
using the "partial credit model" of item response methodology (Adams & Khoo, 1992; 
Masters, 1982). For detailed accounts of the development of such profiles, see Griffin 
(1990), Griffin <& Nix (1991) and Rowe (1992). For a brief account of the related 
technical details for item response methodology see Appendix 5. 

These profiles have been validated against widely used criterion/domaia-referenced 
tests and have been shown to have high levels of face-validity, internal consistency and 
inter-rater reliability. Such profiles have a particular advantage in a study of the 
present kind since they: 
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» provide a means of recording student achievement from the fiill range of both 
formal methods of assessment (tests and related assessment tasks) and informal 
methods (observations and descriptive judgements), typically used by teachers, 
and 

• allow meaningful comparisons to be made across the years of schooling and over 
time. 

Teachers rated each student's level of achievement against the indicators in each of the 
nine Bands (A - 1) of the Reading, Writing and Spoken Language profiles and in each 
of the twelve Levels (1-12) of the Number and Space profiles. Teachers were asked 
to indicate a '3' if all of the behaviours associated with a given band/level were 
consistently displayed by the student, '2' if most of the behaviours were present, '1' if 
some of the behaviours were beginning to be developed, and '0' if none of the 
behaviours had been observed. These ratings for each band/level were then added 
together to give a total score out of 27 for each of the English profiles, or 36 for both 
Mathematics profiles. A weighted composite score was then constructed for English 
achievement using the Reading, Writing and Spoken Language profile scores and 
Mathematics achievement using the Number and Space profile scores. 



Student behaviour 

Three dimensions of student behaviour were measured, namely Sociable/Anti-Social 
Attentive/Inattentive and Settled/Restless, using an instrument developed by Rowe and 
Rowe (1989). While both parent and teacher ratings were obtained, only those of 
teachers have been used in the analyses contained in this report. 

Student attitudes 

Students were asked to indicate their attitudes towards English and mathematics on 
three items: (1) "Do you like reading?", (2) "Do you like writing?", and (3) "Do you 
like maths?" - each measured on 5-point ordinal scales ranging from "Not at all" to 
"Very much". A weighted composite for Attitudes to Learning was computed from 
students' responses on these items. Students' perceptions of the "Quality of School 
Life" were obtained from responses to a 14-item inventory adapted from instruments 
developed by Ainley, Reed & Miller (1986) and Ainley, Goldman & Reed (1990) that 
built upon the scales developed earlier by Williams & Batten (1981). Weighted 
composite scores on four scales were obtained, namely. Liking for School Social 
Acceptance (by peers), Teacher Responsiveness and Curriculum Usefulness, 

Homework 

On six items, students indicated the frequency with which they undertake homework in 
English, mathematics and other subjects - each measured on 5-point ordinal scales 
ranging from "Every day", "2-3 times per week", "Once per week", "2-3 times per 
month or Sometimes", and "Never". Weighted composites for English Homework and 
Maths Homework were computed. 
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Parent responses 



On a Parent Questionnaire, parent(s)/guardian(s) provided data for measures in six 
major areas: (1) family socio-educational background factors (see above);^ (2) 
estimates of the amount of time students spend on school-related and extra-curricula 
activities per week; (3) the source and frequency of help given to students with regard 
to homework; (4) the type and frequency of participation in school activities related to 
curriculum assistance {Assisting with Class Activities), attendance at both parent- 
teacher interviews and information evenings {Attendance at Parent-Teacher 
Interviews, etc\ and school administration {Involvement in Decision-Making), (5) an 
evaluation of the school in terms of the quality of information received on students' 
educational progress and the perceived extent to which the school meets the 
educational need of their child {Parent Perceptions of the School) and, finally, (6) a 
rating of their child's behaviour at home, using the same instrument as that completed 
by teachers (see above). 



Teacher affect 

Measures were obtained from responses on a semantic differential instrument 
consisting of 34 ^even-point evaluative scales. This instrument was adapted from the 
Professional Self-Perception Questionnaire (PSPQ), originally developed by Elsworth 
& Coulter (1977) and modified by Rowe & Sykes (1989). Two scales from the PSPQ 
were used in this study, namely, Energy/Enthusiasm and Warmth towards students. 



Teachers' perceptions of their work environment 

Work environment measures were derived from teachers' responses on the School 
Organisational Health Questionnaire developed by Hart, Conn & Carter (1992). This 
54-item instrument was specifically developed by the Department of School Education 
(Victoria) to obtain valid and reliable measures of 12 related domains of 'school 
organisational health', namely: Morale, Student Orientation, Feedback Work 
Demands, Discipline Policy, Leadership (administrative) Support, Peer Support, Role 
Clarity, Curriculum Co-ordination, Professional Development, Goal Congruence and 
Decision Making, 
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RESULTS FROM PHASE 1 



The size and nature of the sample design allow accurate estimates of distributions to 
be reported for each of the key variables of interest. These are a valuable outcome of 
the study in their own right, as they provide a detailed description of students and 
teachers in Victorian schools for years Prep, 2, 4, 7 and 9. They also provide a point 
of reference for the participating schools in reflecting upon their own situation. These 
results are summarised in Tables 3.1 to 3.7 of Appendix 3. 



Student achievement 

Table 3.1 summarises achievement ievels of students in each of Years Prep, 2, 4, 7 and 
9 on each of the three measures of English and the two measures of Mathematics. 
These data are also summarised graphically in Figures 2 and 3 by means of 'box and 
whisker' plots. The shaded boxes in the graphs represent the range of achievement of 
the middle 50 per cent of students, with the bottom of each box being drawn at the 
level of students achieving at the 25th percentile, and the top of each box showing the 
75th percentile. The star in the middle of each of the boxes represents the level of 
achievement of students at the 50th percentile. The bottom whisker shows the level of 
achievement of the 10th percentile, while the top whisker shows the 90th percentile. 
Lines of best fit have been drawn on each graph for the 10th, 50th and 90th 
percentiles. 

The graphs for the three English profiles indicate a period of rapid growth during the 
first few years of schooling, coinciding with the period during which young people 
acquire basic literacy skills. Thereafter, the graphs indicate a consistent rate of growth 
up to Year 9. It is noticeable, however, that the range of achievement increases 
markedly over the years of schooling with more than four Band widths separating 
Year 9 students at the 10th and 90th percentiles. 

Of particular concern is the flattening out of the growth trajectory at the 10th 
percentile, where the graphs indicate a trend of less than one band width of growth 
between Years 4 to 9. The graphs also provide evidence of a discontinuity between 
primary and secondary schooling, for reading and spoken language especially, with a 
dip in the rate of progress of students in the first year (Year 7) of secondary school. 
This pattern has been observed frequently in previous studies using common measures 
over primary and secondary schooling. 

The graphs shown in Figure 3 for Number and Space indicate steady growth fi-om 
Years Prep to 9, with little evidence of any flattening out of achievement levels among 
the bottom end of the distribution. The graphs give the impression of a steeper rate of 
learning, but this is due to the fact that the mathematics profiles are organised into 12 
Levels rather than nine Bands. As with the English profiles, the spread of achievement 
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Figure 3, Mathematics achievement levels 



increases as students move through the school, with at least four Levels separating 
students at the 10th and 90th percentiles in Year 9. 

For both English and Mathematics, the results as depicted in Figures 2 and 3 are best 
interpreted by referring back to the detailed set of indicators associated with each 
Band/Level for each of the profiles. In this way, it is possible to provide a detailed 
picture of the kinds of abilities possessed by individual students at each Year level, as 
well as to a criterion- or standards-referenced description of Statewide achievement 
levels to complement norm-referenced interpretations. 



Student behaviour 

Table 3.2 gives teachers' ratings of the behaviour of primary and secondary students. 
These results are summarised in the "box and whisker plots" of Figure 4. The plots 
reveal a skew to the right-hand or positive side of the bi-polar scales, with teachers 
recording generally favourable ratings of student behaviours. That is, teachers 
perceive most of their students to be more sociable than anti-social, more attentive 
than inattentive and more settled than restless. 
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Figure 4. Student behaviour 
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The whiskers to the left of the plots indicate, however, a tendency for teachers to rate 
up to 25 per cent of their students towards the inattentive and restless ends of those 
scales. Somewhat unexpectedly, primary and secondary teachers record similar ratings 
of their students, despite the generally held perception that negative student behaviour 
is greater in secondary schools. 



Student attitudes 



Table 3,3 gives results on various measures of primary and secondary students* 
perceptions of school life. These results are displayed graphically in Figure 5, 
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The most noticeable feature of these data is the strong positive attitudes towards 
school life among primary school students. The great majority of primary students like 
school, are accepted socially by their peers, get on well with their teachers, view the 
curriculum as being useful and relevant and like doing school work. 

On the other hand, secondary students, while recording positive responses overall, are 
less favourably inclined towards their experience of schooling. In particular, there are 
significant numbers who do not like school and who believe their teachers do not listen 
to them. 



Home}i>ork 

Table 3.4 sets out results for three measures associated with homework. The first two 
relate to the frequency of homework done in English and mathematics and the third to 
the frequency of help received at home from parents, relatives and friends. Once 
again, these results are shown graphically in Figure 6 for both primary and secondary 
students. 
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Figure 6. Homework and home help with homework 
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The "box an^J whisker" plots indicate great variation in the frequency with which 
homework is aone by primary school students, although it is also evident that primary 
students frequently receive home help with homework. At the secondary level, the 
most noticeable feature is the relatively low frequency with which English homework 
is done compared to the frequency of mathematics homework. At the secondary level, 
students receive home help less frequently with their homework. 

Parent responses 

Table 3.5 summarises parent responses to questions regarding their attendance at 
parent/teacher interviews and information nights, the extent of their involvement in 
decision-making at the school and parent perceptions of the school and the quality of 
the education provided. The results are also depicted graphically in Figure 7. 
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Figure 7. Parent participation and perceptions of the school 
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The results indicate that assistance with class activities is largely a primary school 
phenomenon and that parents of primary school children are very regular attenders of 
meetings at which they receive information on their children's progress, whereas at the 
secondary level there is a noticeable falling off in attendance at such meetings. 
Involvement in school-level decision-making is inevitably limited to a few parents, but 
more are involved at the primary level, perhaps reflecting the greater opportunities in 
primary school because of their smaller size. 

Parents' perceptions of their children's school ave generally positive, confirming the 
commonly observed finding that parents tend to rate highly the school their children 
attend, whatever their views on schools at large or the system generally. 



Teacher affect 

Table 3.6 and Figure 8 give results on two measures of teacher affect, namely their 
energy or enthusiasm and their feelings of warmth towards students. Teachers 
generally have positive feelings of warmth towards students. However, in regard to 
their levels of energy or enthusiasm, despite positive ratings overall, there is a wide 
spread, with significant numbers 25%) rating themselves at the lower end of the 
scale. 



Teachers* perceptions of their work environment 

Table 3.7 presents the results ft*om the measures of teachers' perceptions for 12 
aspects of their work environment. These results are also displayed graphically in 
Figure 9. The "box and whisker" plots reveal that most teachers are generally positive 
about their work environment. However, they indicate that the majority of teachers 
receive little feedback on their work performance. Despite the generally positive 
responses across the range of measures relating to teachers' work environment, there is 
nonetheless considerable variation among teachers on each of the measures. 
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Relationships among student factors 



To examine the relationships among factors operating at the student level, structural 
equation models were fitted to the correlations and covariances among composite 
scores for each of the student-level variables (see Cuttance & Ecob, 1987). This 
resulted in a series of four models, two for primary students and two for secondary 
students, which were judged as providing both good fit to the data and a set of 
relationships which make substantive sense. These models are summarised in Figures 
10 and 11. In these diagrams, each of the arrows represents a statistically significant 
eflfect of one factor on another. The thickness of the arrows indicates the relative size 
of the effects. Table?; 4.1 to 4.4 of Appendix 4 provide estimates of the direct, indirect 
and total standardised path coefficients represented by the diagrams. The related 
technical notes are provided in Appendix 5. 

Primary students 

Consider first Figure 10, Tables 4.1 and 4.2, which relate to primary students. The 
following patterns emerge. 

• Student background characteristics appear to have a relatively small effect on 
achievement levels. 

• The effeu, either direct or indirect, of the socio-educational level (SEL) of the 
student's family on English and mathematics achievement is weak, although 
students from higher SEL levels are more attentive in the classroom and their 
parents are more likely to attend parent/teacher interviews and information nights. 

• Girls have more positive attitudes to learning, are more attentive, and have higher 
English achievement than boys. The effects of gender on attitudes to learning 
include substantial indirect effects through attentiveness. 

• Students from non-English-speaking backgrounds are more likely to come from 
lower socio-educational level families and their parents are less likely to attend 
parent/teacher interviews and information nights. 

• Parent participation in parent/teacher interviews and information nights and doing 
regular homework have positive effects on English achievement and on attitudes to 
learning. 

• Frequent English homework has a positive effect on attitudes to learning. 

• Students' perceptions of the usefijlness of the curriculum strongly affect their 
attitudes to learning. 

• Attentiveness in the classroom has a positive effect on student attitudes to learning 
and on their achievement. In the case of English achievement, this effect is massive. 




Figure 10. Relationships among factors for primary students 
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Figure 11. Relationships among factors for secondary students 
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Secondary students 



Tables 4,3 and 4,4 and Figure 1 1 show the relationships among student-level factors for 
secondary students, which may be summarised as follows, 

• Student background characteristics have an important effect on English achievement, 
although the effect on mathematics achievement is negligible, 

• In particular, the socio-educational level (SEL) of the student's family has a large 
direct effect on English achievement, as well as much smaller effects on a range of 
other student-level factors such as parent participation and attitudes to learning, 

• Gender (female) is a powerflil factor with girls having more positive attitudes to 
learning, more frequently doing English homework, being more attentive in English 
classes and achieving higher levels of English, 

• Students from non-English-speaking background are more likely to come from 
families in the lower socio-educational levels, but do homework more frequently, 

• Living in a rural location has generally negative effects, especially on homework and 
attitudes to learning. These effects are notably more marked for English than for 
mathematics, 

• Parent participation in parent/teacher interviews and information nights has a small to 
negligible effect at the secondary level, 

• Students' perceptions of the extent to which they are socially accepted by their peers 
have a positive effect on their attitudes to learning, 

• Students' attitudes to learning have a strong positive effect on their attentiveness and 
their English achievement, 

• Attentiveness in the classroom has a positive effect on achievement and, as with 
primary students, the effect on English achievement is massive. 



Relationships among teacher factors 



Figure 12 summarises the relationships among various measures of teachers' perceptions 
of their work environment, their levels of energy and feelings of warmth towards 
students. The magnitude of direct, indirect and total effects of key factors are 
summarised in Table 4,5 of Appendix 4, 

Taken together. Figure 12 and Table 4,5 point to the following conclusions, 

• Leadership support is overwhelmingly important in establishing a positive working 
environment for teachers. It is associated with powerful effects on teacher 
involvement in professional development activities, on teachers' perceptions of the 
amount of peer support they receive, on the degree of goal congruence among staff 
within the school, on their involvement in decision-making and on their role clarity. 



Figure 12, Relationships among teacher factors 



• Participation in professional development activities has a positive effect on teachers* 
levels of energy, which in turn affects their feelings of warmth. It is also associated 
with positive effects on their perceptions of the amount of feedback they receive. 
Finally, participation in professional development activities leads to a decline in 
teachers' perceptions of the work demands placed upon them, 

• Teachers' perceptions of the degree of goal congruence within the school have a 
strong effect on levels of teacher morale, student orientation and curriculum co- 
ordination within the school, as well as having a considerable effect on teachers' 
perceptions of the school's discipline policy. There is also a substantial effo rt of 
curriculum coordination on the school's discipline policy. 

Of the various indicators of the quality of teachers' work environment, considerable 
interest surrounds that of teacher morale. To ascertain the most important influences on 
teacher morale, a series of multi-level regression models were fitted to the teacher-level 
data. The model providing the best explanation for teacher morale is summarised in 
Figure 13. Standardised regression coefficients and other details of this model are given 
in Table 4,6, while additional technical information is given in Appendix 5, 

The regression analysis indicates that a model which has as explanatory variables each 
teacher's perceptions of peer support, goal congruence and leadership support, together 
with the average ratings of leadership support of aU teachers in the school, accounts for 
73 per cent of the variance in teacher morale. In this model, the predictor exerting the 
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greatest influence is goal congruence. Leadership support also exerts a strong influence, 
both as a teacher-level and a school-level predictor. 




Figure 13. Predictors of teacher morale 



The relative importance of school, teacher and student effects 

In this research study, the primary aim is to identify the characteristics of schools in 
which students make rapid and sustained progress in English and Mathematics, after 
adjusting for their initial levels of achievement. This involves obtaining longitudinal data 
on achievement in order to compare rates of progress against initial achievement. 

At the conclusion of Phase 1 of this study, only a single measure for each of the student 
achievement outcome measures is available and for this reason the results presented here 
are necessarily tentative. It is nonetheless possible to obtain a preliminary estimate of the 
relative importance of school-, class/teacher- and student-level factors by undertaking 
multi-level analyses using achievement scores adjusted for relevant intake characteristics. 
As noted earlier, there are problems with this approach since it almost certainly results in 
serious underadjustments for the effects of intake characteristics. It nevertheless 
provides a point of comparison with oiher studies reported in the literature. Tables 4.7 
and 4.8 below summarise the results of the multi-level analyses using this approach, 
while Appendix 5 provides the relevant technical details. 

Table 4.7 indicates that when a two-level model is fitted to the achievement data, school- 
level effects account for between 6.6 and 1 1.0 per cent of the total variance, or between 
7.2 and 1 1.4 per cent when intake adjusted scores are used. This finding is very much in 
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line with effect sizes reported in the international research literature. The table also 
indicates that school effects are generally more significant than intake characteristics, 
which account for between 3.7 and 8.8 per cent of the variance. The intake 
characteristics used to adjust students' achievement scores were gender, socio- 
educatioml level non-English-speaking background and rural/non-rural residential 
location. 



Table 4,7 Per Cent of Variance Explained by Student and 
School Level Effects: Two-Level iVIodel 



Level of 
Schooling 
(Outcome) 


Unadjusted Scores 


Intake 
Adjusted Scores 


Intake 
Effects 


Other 
Student 
Effects 


School 
Effects 


Student 
Effects 


School 
Effects 


Primary 
(English) 


6.1 


85.3 


8.6 


90.8 


9.2 


Primary 
(Mathematics) 


3.7 


85.3 


11.0 


88.6 


11.4 


Secondary 
(English) 


8.8 


84.6 


6.6 


92.8 


7.2 


Secondary 
(Mathematics) 


5.4 


86.5 


8.1 


91.4 


8.6 



When teacher/class effects are taken into account and a three-level model fitted to the 
data, a very different picture emerges (see Table 4.8 and Figure 14). School-level effects 
reduce to an almost negligible 0-3.4 per cent of the total variance, whereas teacher/class 
effects represent a massive 28-46 per cent. This finding would be of little surprise in 
situations where schools streamed their students by ability. However, in the Victorian 
context streaming is almost entirely absent, especially in primary schools, in Government 
and Catholic secondary schools and in most independent secondary schools. The 
magnitude of the effects for teacher/class influences thus almost certainly represent the 
impact of class/ teacher factors rather than effects related to the composition of classes. 

Very few comparable studies have been reported in the international literature that have 
used three-level models to simultaneously examine classroom and school effects on 
student achievement. A notable exception is a re-analysis of the EEA second 
mathematics study by Scheerens et al (1989). The results of this re-analysis of 
mathematics achievement of students in their second year of secondary schooling in nine 
countries suggests that the Victorian results compare closely to the pattern of results in a 
number of other countries and are almost identical to those for Finland, New Zealand 
and Sweeden when results based on intake-adjusted scores are compared. Table 4.9 
shows the Victorian secondary mathematics results together with the results for six of 
the nine countries with a similar pattern of results. 
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Table 4.8 Per Cent of Variance Explained by Student, Class/Teacher and 
School Level Effects: Three-Level Model 



l^VCi Of 

Schooling 
(Outcome) 


Unadjusted Scores 


Intake 
Adjusted Scores 


Intake 
tiiects 


Other 
otuoent 
Effects 


Class/ 
Teacher 
Effects 


School 
Jbnccis 


Other 
oiuaeni 
Effects 


Class/ 
1 eacner 
Effects 


School 
unecis 


Primary 

(£#ngllSnj 


O.J 






Z. 1 


D*t.*t 






Primary 


3.6 


51.0 


44.0 


1.4 


52.9 


45.6 


1.5 


Secondary 
(English) 


1.1 


63.2 


26.0 


3.1 


68.5 


28.1 


3.4 


Secondary 
(Mathematics) 


4.3 


58.6 


37.0 


0.0 


61.3 


38.7 


0.0 




Figure 14. Percentage of variation in students* English and mathematics 
achievement due to school effects, class/teacher effects 
and within-student effects 

(Note: Variance estiinalcs for achievement have been adjusted for the student-level intake factors of 
gender, socio-educational levei Xon-English speaking backgound and rural location). 
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Table 4^9 Per Cent of Variance Explained by Class/Teacher and School 
Level Effects: Three -Level ModeU Seven Countries 



Country Class/Teacher School 
Effects Effects 



Sweden 


45 


0 


New Zealand 


42 


0.4 


Finland 


41 


0.2 


Victoria 


39 




(Aust.) 


0 


Scotland 


31 


5 


Luxembourg 


29 


15 


USA 


45 


9 



Source: Scheerens a/., (1989, p. 794) 



In view of the magnitude of teacher effects as they relate to achievement, multi-level 
analyses were then undertaken of teachers' perceptions of their work environment to 
ascertain the proportion of variance due to school effects. Results for the measures of 
teachers* work environment associated with large school effects are summarised in 
Figure 15. The graph indicates that between-school differences are more marked in 
primary than in secondary schools, although for both primary and secondary teachers, 
what school they teach in has a big effect on several aspects of their perceptions of their 
work environment. For both primary and secondary teachers, between school 
differences are most marked for teacher morale, with school effects accounting for 33 
per cent of the variance of primary teachers' ratings and 31 per cent of secondary 
teachers' ratings. 
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Figure 15. Per cent of variance in teachers' perceptions of their 
work environment due to schooMevel effects 
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KEY FINDINGS FROM PHASE 1 



The data from Phase 1 of the project provide a portrait of schooling that is illuminating 
in its own right, as well as containing vital clues for school improvement and information 
relevant to policy issues. The following ten key findings have been identified as being of 
special interest and relevance at the present time. 



1. Profiles provide an effective frame%vork for monitoring and reporting 
achievement 

An important conclusion which can be drawn from Phase 1 is that the Victorian Profiles 
function extremely well as a framework for monitoring student progress over Year Prep, 
to Year 9. This finding is important in the context of national initiatives to develop 
subject profiles similar to the Victorian Profiles and for school systems as they consider 
approaches to accountability and school improvement. The performance data obtained 
indicate clearly what students can typically do at different Year levels and how the level 
and range of achievement increases over the years of schooling. 

At the school level, the indications are that most teachers feel very comfortable using the 
profiles and with the information they provide. The exception is secondary teachers of 
mathematics, who are generally less familiar with or less at ease in using the mathematics 
profiles, and as a consequence, did not always return completed profile assessments of 
their students. This is not surprising, since at the time of data collection they had 
received little by way of formal in-service training in their use. This is an issue that 
needs to be addressed during 1993 for the participating schools and perhaps more 
broadly. 

2. Schools are not without considerable influence in overcoming inequalities 
imposed on students by their background characteristics 

The report of the first major empirical study of school effectiveness. Equality of 
Educational Opportunity concluded: "...that schools bring little influence to bear on a 
child*s achievement that is independent of his background and general social context" 
(Coleman et ai, 1966, p. 325). The student-level analyses of the Phase 1 data can be 
seen as encouraging and confirming one of the key conclusions of recent school 
effectiveness research, namely that the effects of home background are not as pervasive 
as was indicated in this early study. 

Certainly, home background characteristics have an impact, for example on secondary 
English achievement, but their effects are often not direct, but mediated through other 
factors over which schools have a measure of influence, such as attendance at 
parent/teacher interviews and information nights (in the case of primary English). Some 
characteristics, such as non-English speaking background, which is often assumed to be 




negative in its impact, have positive effects, for example, on attitudes to learning and 
frequency of homework. In addition, effects can be quite complex. For example, it is 
evident that intake characteristics such as family socio-educational level, gender and 
rural location exert a greater influence on English than on mathematics achievement. 
This indicates that effects can be quite specific within the curriculum. 

More work needs to be done on examining the effects of home background 
characteristics and especially on the compounding effects for those students having 
membership of more than one educationally disadvantaged group. Such work has 
relevance for decisions about where to target special needs funding and approaches to 
the education of students with special needs. 

3. The bottom decile of students makes minimal progress in English beyond Year 4 

The graphs showing the distribution of achievement for each of Years Prep, 2, 4, 7 and 
9, point to the fact that most students make consistent progress over the years of 
schooling, despite evidence of a slight discontinuity between primary and secondary 
school. Also, the spread in achievement levels increases the longer students remain in 
school. The most significant finding, however, is evidence of a trend for the bottom 
decile of students to make minimal progress in reading and writing between Years 4-9. 

This finding highlights the crucial importance of the eariy childhood years in establishing 
a firm foundation for learning, especially given the parallel findings regarding the 
relatively strong effects of home background factors on students' English achievement at 
the secondary level. One conclusion to be drawn from these results is that they confirm 
the views of those who advocate placing a high priority on eariy intervention programs 
such as Reading Recovery which have the potential to place students on a growth 
trajectory characterised by consistent progress over the years of schooling. Another 
conclusion is that the later years of schooling are less effective in overcoming any 
disadvantage relating to student background characteristics. 



4. Attentiveness has a massive effect on student achievement 

Teachers have generally positive perceptions of the behaviour of students, with negative 
behaviour being seen as characteristic of a minority of students. However, in the context 
of fitting explanatory models, it is the attentiveness aspect of behaviour that is the most 
salient in explaining student achievement. This finding is consistent with a large clinical 
research literature that shows a strong relationship between students' learning outcomes 
and their attentive/inattentive behaviours. The other dimensions of behaviour measured 
in the study, namely sociable/anti-social and settled/restless, are unrelated to other 
student-level factors and in particular to levels of achievement. 

At this stage in the study, it is not possible to say much about the underiying causes of 
inattentiveness or the extent to which attentiveness can be enhanced by the teacher and 
the school. There are grounds for believing that in the majority of cases of 
inattentiveness, psychosocial factors such as anxiety or stress may be the most common 
explanation while, for a minority, inattentiveness may be related to auditory or other 
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perceptual problems, or to deeper-seated learning difficulties. These are issues for 
further investigation during the remaining two phases of the project. 



5, The frequency with which students do homew^ork varies greatly between 
students; the emphasis on homovork changes from primary to secondary school 

At the primary school level, homework has a positive influence on students' attitudes 
towards learning and to a lesser extent their attentiveness in the classroom and 
perceptions of the quality of school life. However, the most significant aspect of the 
data relating to homework is the very different frequencies with which primary and 
secondary students do homework. Students in primary school appear to do more 
homework in English than in mathematics, whereas the pattern is reversed for secondary 
students. In both primary and secondary schools there is enormous variability in the 
frequency with which students do homework and this applies especially to English 
homework. These results suggest that for some schools it may be appropriate to review 
their policies to ensure a more positive and consistent approach to this aspect of the 
relationship between the home and the school. 



6. Parents' involvement in monitoring their child's education is important for 
primary students 

The parent response data indicate that there are lower levels of parent participation at 
the secondary level than at the primary level and that, at both primary and secondary 
levels, parents have generally positive perceptions of their children's school. These 
findings are consistent with the related research. In modelling the relationships among 
student-level factors, it was found that at the primary school level parent involvement in 
parent/teacher interviews, information nights and other activities connected with 
monitoring their child's progress has a positive effect on student attitudes, behaviour and 
learning. This finding is significant in that it suggests school intervention strategies 
which could increase this kind of participation. 



Z Teachers are generally positive about their work environment, but they receive 
little feedback on their work performance 

The consistently positive ratings of their work environment by teachers is of interest at a 
time when it is often asserted that morale is at an all-time low. Teachers completed the 
teacher questionnaire during Term 3 of 1992, which was generally regarded as a stressful 
time during a particularly diflTicult year, especially for the many teachers involved in the 
process of applying for promotion to the recently-created Advanced Skills Teacher 
positions. The exception was their ratings of the amount of feedback they receive on 
their work performance. This suggests that systematic consideration of the whole issue 
of teacher appraisal is long overdue. 
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& Leadership support is critically important in establishing a positive teacher work 
environment 

The finding regarding the overwhelming importance of leadership support in establishing 
a positive work environment accords with the school effectiveness research literature 
generally. At this stage, however, the study data include only teachers* responses on 
leadership support. During 1993 it is proposed to introduce a school questionnaire to be 
completed by the principal and the leadership team, designed to explore perceptions of 
their personal leadership style, and the issues they confront in exercising leadership, in 
administering the school and in managing staff. This approach would seem to be 
especially relevant in view of local initiatives aimed at increasing school autonomy. In 
the meantime, the finding of large amounts of between-school differences in teacher 
morale, and the capacity to predict 73 per cent of the variation in teacher morale in terms 
of leadership support, goal congruence and peer support, suggest that improving the 
quality of leadership support in the school is the key to positive teacher work 
environments. 

Taken together with th finding regarding the importance of professional development 
on increasing teachers' energy, warmth and perceptions of feedback, and on decreasing 
their perceptions of work demands, the teacher response data yielded usefial indications 
of how schools and school systems might usefijlly direct their efforts to improve teacher 
quality. Systematic professional development programs for teachers and leadership 
training programs for principals, vice-principals and senior staff stand out as areas which 
may demand special priority. 

9, The key to improved educational outcomes is teacher effectiveness 

As observed throughout the report, an ultimate objective of the project is to identify 
characteristics of schools that are effective given the intake characteristics of their 
students. At this stage, in the absence of longitudinal data, it is not possible to identify 
which are the effective schools in the study, nor what their characteristics are. On the 
basis of intake-adjusted scores, however, one can begin to form some assessment of the 
relative importance of student, class/teacher and school effects. 

Analyses of the Phase 1 data cast new light on the key findings of much previous 
research and emphasise the importance in studies of school effectiveness that allow 
class/teacher effects to be taken into account (c.f Scheerens et ai, 1989). In the present 
study, class/teacher effects were found to be substantial, accounting for between 28.1 to 
45.6 per cent of the variance in student achievement. 

This finding underscores the fact that learning takes place in classrooms through the 
interaction of students and their teachers Interpreting these findings requires some care, 
however, since they have several possible explanations. One is that they reflect 
streaming of classes. This explanation can almost certainly be discounted since mixed 
ability classes have for some time been the norm in Victorian schools. 

A second explanation is that these results reflect inconsistent use of the profiles by 
teachers. Here again, there is little evidence in support of this explanation, as the 
profiles have been demonstrated to have high inter-rater reliability. In addition, efforts 
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were made to identify and remove from the analyses outlier results submitted by 
teachers. A very small number of such cases was identified. Nevertheless, this 
explanation is something to be followed up in phases 2 and 3 of the project by 
introducing student response measures of achievement to validate teacher assessments 
using the profiles. 

A possible tliird explanation, namely that the finding of large class/teacher effects and 
small to insignificant school effects over and above those attributable to teachers, is a 
reflection of variations in teacher quality. This explanation fits with the findings of a 
study of Victorian primary schools by Ainley, Goldman & Reid (1990), which found that 
differences among teachers within schools were greater than differences among schools 
in the growth they achieved in their students. It suggests that it is primarily through the 
quality of teaching that effective schools make a difference. Clearly, this hypothesis will 
be a major focus of the remaining phases of the project. 



10. Schools find data obtained through participation in the project to be useful in 
the context of school improvement 

One of the aims of the Victorian Quality Schools Project has been to facilitate school 
improvement processes within participating schools. To this end, at the 22 separate 
meetings with staff from participating schools, attention has been devoted to exploring 
ways in which each school's results might be used both in the context of traditional 
approaches to school improvement and more modem approaches to quality assurance 
and total quality management. In particular, emphasis has been placed on exploring the 
relevance of the information provided as a result of participation in the project with an 
approach to quality assurance that emphasises continuous improvement of processes and 
outcomes, understanding the reasons for variation, measurement of processes and basing 
decisions on facts and data, and establishing a client orientation. 

The overwhelming reaction from schools is that they value highly feedback on their own 
school's pattern of results. In many instances it provides data of a kind that the school 
has not had access to in the past and provide new insights or confirmations of trends that 
have been suspected but not confirmed. Staff from the participating schools also 
respond positively to the overall findings of the project and are interested in relating their 
own results to these findings. 

Considerable emphasis has also been placed on the need to handle school results with 
great sensitivity and due consideration of the need to protect the anonymity of 
individuals. The dangers of misinterpreting results has likewise been stressed and 
alternative explanations are explored to sensitise staff to the range of valid interpretations 
for a particular result. 

The longitudinal nature of the project means that there is an opportunity to monitor the 
impact of involvement in the project on school improvement processes within the 
participating schools. 




Towards a model of school and teacher effectiveness 



While the literature on school efTectiveness contains a number of Vecipe* style lists of 
characteristics of effective schools (e.g., Edmonds, 1981; Levine & Lezotte, 1990; 
Purkey & Smith, 1983), there are fewer instances of attempts to place empirical findings 
within a theoretical framework that shows the relationships among key factors. Some 
notable exceptions include models proposed by Banks (1992), Coleman & Collinge 
(1991), Cuttance (1992), Mortimore (1992), Reynolds, Hopkins & Stoll (1993) and the 
research agenda proposed by Scheerens (1993). The findings of this study add 
significantly to u.e available empirical evidence. The final section of this report attempts 
to bring together ideas and evidence in the literature with the present findings to propose 
a tentative model of teacher and school effectiveness that will be tested and refined 
during the remaining phases of the study. 

At this stage, it is suggested that any attempt to embrace both school and teacher 
effectiveness in a single theoretical model will have the followmg characteristics: 

• It must be a highly generalised model to accommodate the range of relevant factors 

• It must reflect the organisational charactenstics of schools and the hierarchical 
clustering of students within teaching groups, within schools 

• It must allow for complex interrelationships at each level of factors, as well as 
between levels. 

• It must be dynamic and reflect the fact that learning takes place over time within a 
context that is constantly and often dramatically changing. 

Given these characteristics, the model summarised in Figure 16 is offered as a heuristic 
of school and teacher effectiveness. An essential feature of the model is the recognition 
of the importance of student readiness or availability for school learning (Rutter, 1985). 
Major elements of this availability include functional levels of attentiveness and positive 
attitudes towards learning. These elements are influenced, both directly and indirectly by 
a stable and supportive psychosocial home environment and by parent participation and 
interest in monitoring their child's educational progress. They are also influenced by 
short-term critical events, including illness, absence from school, and a range of family 
and personal trauma events, which can include family breakup, loss of employment, 
death or illness of close relatives, and so on (Wallerstein, 1991). The other key factors 
influencing readiness and availability for learning are positive teacher affect (Ashton & 
Webb, 1986), teachers' positive perceptions of their work environment and, of course, 
the quality of teaching. School organisation and leadership support are vitally important 
in establishing a context for positive teachers' perceptions of their work environment. 
The dynamic aspect of the model is indicated by its conception of student learning 
outcomes as being represented by the rate and consistency of progress made by students. 

It will be noted that at this stage, the present study provides little or no evidence 
regarding the effects, both direct and mediated, of critical events affecting students or of 
the quality of teaching. These are factors to be incorporated into phases 2 and 3 of the 
study that will now be shaped in order to test the validity, reliability and stability of this 

39 



32 



model, to elucidate the conditions and extent to which students, parents, teachers and 
schools make a difference, and provide practical strategies for school and teacher 
improvement designed to maximise and sustain student growth. 



school organisation 
and leadership 
support 




Figure 16, A heuristic model of school and teacher effectiveness 



340 



REFERENCES 



Adams, RJ., & Khoo, S.T. (1992). TITAN: The ACER Interactive Test Analysis System. Hawthorn, 
Vic: The Australian Council for Educational Research. 

Ainley, J., Goldman, J., & Reed, R. (1990). Primary Schooling in Victoria: A Study of Students' 
Attitudes and Achievements in Years 5 and 6 of Government Primary Schools. (Research 
Monograph No. 37). Hawthorn, Vic: The Australian Council for Educational Research. 

Ainley, J., Reed, R., & Miller, H. (1986). School Organisation and the Quality of Schooling: A Study of 
Victorian Government Secondary Schools. (ACER Research Monograph No. 29). Hawthorn, Vic: 
The Australian Council for Educational Research. 

Alwin, D.F., & Jackson, D.J. (1980). Measurement models for response errors in surveys: Issues and 
applications. In, Schuesslcr (Ed.), Sociological Methodology. San Francisco: Jossey-Bass. 

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573. 

Ashton, P., & Webb, R. (1986). Making a Difference: Teachers' Sense of Efficacy and Student 
Achievement. New York: Longman. 

Banks, D.K. (1992). Effective Schools Research: A Multilevel Analysis of the Conceptual Framework. 
Unpublished doctoral dissertation, The University of Melbourne Institute of Education. 

Bock, R.D. (Ed.). (1989). Multilevel Analysis of Educational Data. New York: Acdemic Press. 

Brown, R.L. (1989). Using covariance modeling for estimating reliability on scales with ordered 
polylomous variables. Educational and Psychological Measurement, 49, 383-398. 

Bryk, A.S., & Raudenbush, S.W. (1989). Toward a more appropriate conceptualization of research on 
school effects: A three-level hierarchical linear model. In, R.D. Bock (Ed.), Multilevel Analysis of 
Educational Data (pp. 159-204). New York: Academic Press. 

Castles, I. (1986). Australian Standard Classification of Occupations: Statistical Classification. 
Australian Bureau of Statistics. Canbena: C. J. Thompson Government Printer. 

Chapman, J., Angus, L., Burke, G., & Wilkinson, V. (Eds.) (1991). Improving the Quality of Australian 
Schools. Australian Education Review No. 33. Hawthorn, Vic: The Australian ouncil for 
Educational Research. 

Coleman, J., Campbell, E., Hobson, C, McPartland, J., Mood, A., Weinfield, F., & York, R. (1966). 
Equality of Educational Opportunity. Washington, DC: U.S. Government Printing Office. 

Coleman, P., & Collinge, J. (1991). In the web: Internal and external influences affecting school 
improvement. School Effectiveness and School Improvement, 2 (4), 262-285. 

Cuttance, P. (1992). Evaluating the effectiveness of schools. In, D. Reynolds and P. Cuttance (Eds.). 
School Effectiveness: Research, Policy and Practice. London: Cassell. 

Cuttance, P., & Ecob, R. (Eds.). (1987). Structural Modeling by Example: Applications in Educational, 
Sociological and Behax'ioral Research. Cambridge: Cambridge University Press. 

Dawkins, J.S. (1988). Strengthening Australia's Schools: A Consideration of the Focus on Content of 
Schooling. Canberra, ACT: Australian Government Publishing Service. 



DES (1984). School Standards and Spending: Statisdcal Analysis, A Further Appreciation. London: 
Department of Education and Science. 

Edmonds, R.R. (1981). Making public schools efTcclive. Social Policy, 12,56-60. 

Elsworth, G.R., & Coulter, F. (1977). Aspiration and Attainment: The Measurement of Professional 
Self Perception in Student Teachers, Occasional Paper, No. 11. Hawthorn, Vic: The Australian 
Council for Educational Research. 

Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalized least squares. 
Biometrika, 73, 43-56. 

Goldstein, H. ( 1987). Multilevel Models in Educational and Social Research, Lx)ndon: Charles Griffm 
& Co. Ltd. 

Goldstein, H., & McDonald, R.P. (1988). A general model for the analysis of multilevel data. 
Pychometrika, 53 (4), 455-467. 

Goodlad, J. (1982). A study of schooling: Some implications for school improvement. Phi Delta 
Kappan, 64. 

Goodlad, J. (1983). A Place Called School, New York: McGraw-Hill. 

Griffin, P.E. (1990). Profiling literacy development: Monitoring the accumulation of reading skills. 
Australian Journal of Education, 34, 290-31 1. 

Griflln, P. E., & Nix, P. (1991). Educational Assessment and Reporting: A New Approach, London: 
Harcourt Brace Jovanovich. 

Hart, P.M., Conn, M., & Carter, N. (1992). School Organisational Health Questionnaire: Manual. 
Melbourne: Vic: Department of School Education. 

Jencks, C, Smith, M., Acland, H., Bane, M. J., Cohen, D., Gentis, H., Heynes, B.,& Michelson, S. 
(1972). Inequality: A Reassessment of the Effect of Family and Schooling in America, New York: 
Basic Books. 

Joreskog. K.G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133. 

Joreskog, K.G., & Sorbom, D. (1988). PRELIS: A Program for Multivariate Data Screening and Data 
Summarization: A Preprocessor for LISREL (2nd ed.). Chicago, IL: Scientific Software, Inc. 

Joreskog, K.G., & SCrbom, D. (1989). LISREL 7 User's Reference Guide, Chicago, IL: Scientific 
Software, Inc. 

Lee, V.E., Dcdrick, R.F., & Smith, J.B. (1991). The effect of the social organization of schools on 
teachers' efficacy' and satisfaction. Sociology of Education, 64, 190-208. 

Levine, D.U., & Lezotte (1990). Unusually Effective Schools: A Review and Analysis of Resaerch and 
Practice. Madison, WS:National Centre for Effective Schools Research & Development. 

Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174. 

McDonald, R.P., & Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural 
relations in two-level data. British Journal of Mathematical and Statistical Psvchology, 42, 215- 
232. 

McGaw, B., Piper, K., Banks, D., & Evans, B. (1992). Making Schools More Ejfective: Report of the 
Effective Schools Project, Hawthorn. Vic: The Australian Council for Educational Research. 



36 42 



Morlimore, P. (1991). School effectiveness research: Which way at the crossroads? School 
Effectiveness and School Improvement, 2 (3), 213-229. 

Monimore, P. (1992). Issues in school effectiveness. In. D. Reynolds and P. Cuttance, School 
Effectivness: Research, Policy and Practice, London: Cassell. 

Monimore, P., Sammons, P., & Ecob, R. (1988a). The effects of school membership on pupils' 
educational outcomes. Research Papers in Education, 3 (1), 3-26. 

Monimore, P., Sammons, P., StolU L., Lewis. D., & Ecob, R. (1988b). School Matters, Wells: Open 
Books. 

Munck, M.E. (1979). Model Building in Comparative Education: Applications of the USREL Method 
to Cross-National Survey Data. International Association for the Evaluation of Edcuational 
Achievement Monograph Series No. 10. Stockholm: Almqvist & Wiksell. 

NCEE - National Commission on Excellence in Education (1983). A Nation at Risk: The Imperative for 
Educational Reform. Washington, DC: U.S. Department of Education. 

OECD (1983). Compulsory Schooling in a Changing World. Paris: Organisation for Economic 
Cooperation and Development. 

OECD (1986). Education and Training for Manpower Development. Paris: Organisation for Economic 
Cooperation and Development. 

OECD (1989). Schools and Quality: An International Report. Paris: Organisation for Economic 
Cooperation and Development. 

Prosser, R., Rasbash, J., & Goldstein, H. (1991). MLS - Software for Three-Level Analysis. Institute of 
Education, University of London. 

Purkey. S.C., & Smith, M.S. (1983). Effective schools: A review. Elementarv School Journal, 83 (4), 
427-452. 

Raudenbush, S.W., & Bryk, A.S. (1989). Quantitative models for estimating teacher and school 
effectiveness. In, Bock, R.D. (Ed.), Multilevel Analysis of Educational Data. New York: 
Academic Press. 

Raudenbush, S.W., & Willms, J.D. (Eds.). (1991). Schools, Classrooms and Pupils: International 
Studies of Schooling from a Multilevel Perspective. New York: Academic Press. 

Reynolds, D., & Cutlancc, P. (Eds.) (1992). School Effectiveness: Research, Policy and Practice. 
London: Cassell. 

Reynolds, D., Hopkins, D., & Stoll, L. (1993). Linking school effectiveness knowledge and school 
improvement practice: Towards a s>'nergy. School Effectiveness and School Improvement, 4(1), 
37-58. 

Rowe, K.J. (1990). The Importance of Reading at Home: Research in Victorian Education. Melbourne, 
Vic: State Board of Education. 

Rowe, K.J. (1991a). Students, Parents, Teachers and Schools Make a Difference: A Summary Report of 
Major Findings from the 100 Schools Project - Literacy Programs Study. Melbourne, Vic: School 
Programs Division, Ministry of Education. 



43 

37 



Rowe, K.J. (1991b). The influence of reading activity at home on students' attitudes towards reading, 
classroom attentiveness and reading achievement: An application of structural equation modelling. 
British Journal of Educational Psychology, 61 (1), 19-35. 

Rowe, K.J. (1992). Subject Profiles: What, How and Why? Melbourne, Vic: School Improvement 
Branch, Department of School Education. 

Rowe, K.J., & Rowe, K.S. (1989). Assessing Behavioural Change: The Development and Psychometric 
Properties of a Parent and Teacher-Administered Child Behaviour Inventory for use in 
Educational and Epidemiological Research. Melbourne, Vic: School Programs Division, Ministry 
of Education, and Department of Paediatrics, The University of Melbourne. 

Rowe, K.J., & Rowe, K.S. (1992a). The relationship between inattentiveness in the classroom and 
reading achievement (Part A): Methodological Issues. Journal of the American Academy of Child 
and Adolescent Psychiatry, 31 (2), 349-356. 

Rowe, K.J., & Rowe, K.S. (1992b). The relationship between inattentiveness in the classroom and 
reading achievement (Part B): An Explanatory Study. Journal of the American Academy of Child 
and Adolescent Psychiatry. 31 (2), 357-368. 

Rowe, K.J., & Sykes, J. (1989). The impact of professional development on teachers' self perceptions. 
Teaching and Teacher Education, 5 (2), 129-141. 

Rutter, M. (1985). Family and school influences on behavioural development. Journal of Child 
Psychology and Psychiatry, 26, 349-368. 

Rutter, M., Maughan, B., Mortimer, P., Ouston, J., & Smith, A. (1979). Fifteen Thousand Hours: 
Secondary Schools and Their Effects on Children. Somerset: Open Books. 

Scheerens, J. (1993). Basic school effectiveness research: Items for a research agenda. School 
Effectiveness and School Improvement, 4(1), 17-36. 

Scheerens, J., Vermeulen, C.J.A.J., & Pelgrum, W.J. (1989). Gernerizability of instrunctional snd 
school effectiveness indicators across nations. International Journal of Educational Research, 13 
(7), 789-799. 

Victoria (1991). English Profiles Handbook: Assessing and Reporting Students' Progress in English. 
Melbourne, Vic: School Programs Division, Ministry of Education and Tr^^ning. 

Victoria (1992). Mathematics Profiles Handbook - Number and Space: Assessing and Reporting 
Students' Progress in Mathematics. Melbourne, Vic: School Improvement Branch, Department of 
School Education. 

Wallerstein, J.S. (1991). The long-term effects of divorce on children. Journal of the American 
Academy of Child and Adolescent Psychiatry, 30 (3), 349-360. 

Wcrts, C.E,, Rock, D.R., Linn, R.L., & JCreskog, K.G. (1978). A general method for estimating the 
reliability of a composite. Educational and Psychological Measurement, 38, 933-938. 

Williams, T., & Batten, M. (1981). The Quality of School Life. ACER Research Monograph No. 12. 
Hawthorn, Vic: The Australian Council for Educational Research. 



44 




38 



Appendix 1 



ACHIEVED SAMPLE 



Table 1.1 Student Sample, 1992 



Year Level 


Government 


Catholic 


Independent 


Total 


Prep 


1719 


421 


141 


2281 


Year 2 


1656 


389 


143 


2188 


Year 4 


1651 


402 


156 


2209 


Primary 


5026 


1212 


440 


6678 


Year 7 


2624 


389 


648 


3661 


Year 9 


2576 


391 


603 


3570 


Secondary 


5200 


780 


1251 


7221 


Total 


10226 


1992 


1691 


13909 


Table 1.2 Teacher Sample, 1992 


School Type 


Government 


Catholic 


Independent 


Total 


Primary 


277 


69 


19 


365 


Secondary 


454 


39 


8 


538 


P-12 






65 


65 


Total 


731 


108 


92 


931 




Table 1.3 


School Sam 


pie: 1992* 




School Type 


Government 


Catholic 


Independent 


Total 


Primary 


41 


12 


6 


59 


Secondary 


22 


3 


6 


31 



II is not appropriate lo add logctJier die numbers of primary aiid secondary schools since some are the same 
school, or campuses of the same school. 
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Appendix 2 



MEASUREMENT OF VARIABLES 



Table 2.1 Measurement of Student-Level Variables 



Variable 



Number 
of 
Items 



Reliability 
Prim. Sec 



Description or 
Sample Item(s) 



Student Background 
Socio-Educational Level 



Non-English Speaking 
Background 



Rural/Non-rural 

Student Achievement 
English 

Mathematics 

Student Behaviour 

Anti-Social/Sociai 



Inattentive/Attentive 

Rcstless/Sctei.cd 

Student Attitudes 
Liking for School 

Social Acceptance 



0.908 0.908 Based mainly on parent's occupation 
status but includes parent's level of 
education and an income measure. 

0.969 0.974 Based on whether or not the student, 

mother and/or father was bom in a non- 
English speaking country and whether or 
not English is mainly spoken at home. 

1 .000 1 .000 Based on whether or not the student 
resides in a city of more than 25,000 
people. 



0.961 0.931 Based on the Victorian English Profiles 
(reading, writing and spoken language). 

0.966 0.900 Based on the Victorian Mathematics 
Profiles (number and space). 



0.939 0.931 Can't control 
behaviour 

Irritable/touchy' . 
0.946 0.957 Easily frustrated . 

Aimless activity 
0.926 0.938 Restless 

On the go, lively 



Controls 
behaviour 

Even tempered 

Persistent 

. . Purposeful activity 

Relaxed 

Settled, calm 



0.889 0.915 My school is a place where I get 
enjoyment from being there. 

0.770 0.790 My school is a place where I get on well 
with other students in my class. 
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Table 2.1 (cont) Measurement of Student-Level Variables 

Number Reliability Description or 
Variable of Sample Item(s) 
Items Prim, Sec* 



Teacher Responsiveness 3 0.842 0.807 My school is a place where my teachers 

listen to what I say. 

Curriculum Useftilness 3 0.962 0.894 My school is a place where what I learn 

will be useful. 



Attitude to Learning 4 0.672 0.673 I like reading; I like writing; I like 

maths. 



Homework Home 
Help 

English Homework 4 0.699 

Mathematics Homework 2 0.832 

Home Help with 3 0.593 
Homework 



0.753 I read books to my family every day; I 
talk with my family about books I have 
read every day. 

0.919 I do Maths homework eveiy day. 

0.702 Mother/Father, how often do you help 
your child with homework. 



Parent Responses 



Assisting with Class 
Activities 



0.837 0.921 1/we read to, or listen to students read at 
school; 1/we assist teachers in the 
classroom. 



Attendance at 3 0.771 

Parent/Teacher 

Interviews, etc. 

Involvement in 4 0.901 

Decision-Making 

Parent Perceptions of 3 0.864 

the School 



0.842 1/we attend parent-teacher interviews 
about our child's progress. 

0.920 1/we contribute to decision-making on 
school administration and finance. 

0.854 The school provides quality information 
about the educational progress of 
children; The school is meeting the 
educational needs of children. 



* Scale reliability indices are calculated from the maximally weighted factor score regression 
coefficients obtained from the related one-factor congeneric models. For further details see 
Appendix 5. 




Table 2»2 Measurement of Teacher-Level Variables 



Number Reliability 
Variable of 
Items 



Description or 
Sample Item(s) 



Teacher Affect 

Energy/Enlhusiasm 

Warmth 
Teacher 

Work Environment 
Morale 

Student Orientation 
Feedback 

Work Demands 
Discipline Polic\* 

Leadership Support 

Peer Support 
Role Claritv 



Curriculum 
Coordination 

Professional 
Development 

Goal Congruence 



Decision-Making 



0,9 15 Unenthusiastic Enthusistic 

Inert Energetic 

0.895 Hostile Friendly 

Reproaching Comforting 



0,882 The morale in this school is high. 

0.830 This school promotes the concept of 

students being individuals. 

0.934 There is a structure and ongoing process 

that provides feedback on my work 
performance. 

0.839 Teachers are overloaded with work in 

this school. 

0.871 The rules and sanctions relating to 

discipline in this school are well 
understood by teachers and students. 

0.937 There is support from the administration 

in this school. 

0.924 1 receive support from my colleagues. 

0.806 I am always clear about what others at 

school expect of me. 

0.808 There is eflcctive coordination of the 

curriculum in this school 

0.882 I am encouraged fo pursue further 

professional development. 

0.873 My personal goals are in agreement with 

the goals of this school. 

0.849 There is opportunity for staff to 

participate in school polic>' and decision- 
making. 



* Scale reliability indices are calculated from the maximally weighted factor score regression 
coefficients obtained from the related one-factor congeneric models. For further details see 
Appendix 5. 
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Appendix 3 

RESULTS FROM PHASE 1 



Table 3.1 Student Achievement 



















/Q 


Variable 


lutn 




f nth 


7Sth 

/Oil! 


90th 


Mean 


S.D. 


Conndence 




%-ile 


%-iIe 


%-iIe 


%-iIe 


%-ile 






Limits 


English 


















Reading 
















±0.095 


Prep 


3 


3 


5 


6 


8 


5.19 


2.19 


Yr. 2 


6 


9 


10 


12 


14 


10.01 


2.76 


±0.122 


Yr. 4 


9 


11 


13 


15 


17 


12.85 


3.10 


±0.138 


Yr. 7 


10 


12 


15 


18 


20 


15.06 


3.88 


±0.144 


Yr. 9 


12 


15 


18 


21 


24 


17.67 


4.72 


±0.171 


Writing 
















± 0.097 


Prep 


3 


4 


6 


7 


9 


5.77 


2.22 


Yr. 2 


8 


9 


11 


12 


13 


10.32 


2.32 


±0.103 


Yr. 4 


9 


11 


13 


15 


16 


12.82 


2.98 


±0.132 


Yr. 7 


11 


13 


15 


18 


20 


15.53 


3.77 


±0.140 


Yr. 9 


12 


15 


18 


21 


24 


18.10 


4.34 


±0.157 


Ural 


















Prep 


3 


4 


6 


7 


9 


5.53 


2.32 


±0.101 


Yr. 2 


6 


8 


9 


11 


13 


9.61 


2.69 


±0.119 


Yr. 4 


8 


10 


12 


15 


16 


12.27 


3.17 


±0.141 


Yr. 7 


9 


12 


14 


17 


20 


14.24 


4.13 


±0.153 


Yr. 9 


11 


14 


18 


21 


23 


17.21 


4.87 


±0.177 


Mathematics 


















Number 


















Prep 


2 


2 


3 


4 


6 


3.43 


1.55 


±0.067 


Yr. 2 


6 


7 


9 


10 


12 


8.81 


2.37 


±0.106 


Yr. 4 


10 


12 


15 


17 


19 


14.68 


3.52 


±0.157 


Yr. 7 


16 


19 


21 


24 


27 


21.58 


4.38 


±0.172 


Yr. 9 


21 


24 


27 


30 


33 


27.13 


4.61 


±0.184 


Space 


















Prep 


3 


3 


5 


6 


7 


4.92 


1.80 


±0.078 


Yr. 2 


6 


8 


9 


11 


12 


9.12 


2.36 


±0.106 


Yr. 4 


9 


12 


14 


16 


18 


14.04 


3.43 


±0.153 


Yr. 7 


13 


16 


20 


23 


25 


19.40 


4.66 


±0.183 


Yr. 9 


19 


22 


25 


28 


31 


24.77 


4.82 


±0.193 
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Table 3.2 Student Behaviour 



Variable 


10th 
%-iIe 


25th 
%-iIe 


50th 
%-iIe 


75th 
%-ile 


90th 
%-iIe 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


Anti-Social/ 
Social 

Primary 
Secondary 


-0.35 
-0.36 


0.40 
0.40 


1.15 
1.15 


1.86 
2.00 


2.00 
2.00 


1.01 
1.01 


0.93 
0.98 


±0.023 
± 0.023 


Inattentive/ 
Attentive 

Primar\' 
Secondary 


-0.84 
-0.82 


0.00 
0.00 


0.89 
0.99 


1.59 
1.62 


2.00 
2.00 


0.68 
0.69 


1.08 
1.09 


± 0.026 
± 0.026 


Restless/ 
Settled 

Primarv' 
Secondary 


-0.90 
-0.69 


0.00 
0.00 


0.90 
1.00 


1.79 
1.84 


2.00 
2.00 


0.68 
0.79 


1.12 
1.08 


± 0.028 
± 0.026 



Table 3.3 Student Attitudes 



Variable 


10th 
%-ile 


25th 
%-ile 


50th 
%-ile 


75th 
%-iIe 


90th 
%-iIe 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


Liking for School 
Primarv* 
Secondary 


0.40 
- 1.14 


0,91 
-039 


1.50 
0.47 


2.00 
0.67 


2,00 
1.23 


1.32 
0.17 


0.77 
0.91 


±0.019 
±0.022 


Social 
Acceptance 

Primarv* 
Secondars* 


0.34 
0.27 


0.89 
0.67 


1.38 
1.16 


2,00 
1,73 


2.00 
1.73 


1.28 
1.04 


0.73 
0.70 


±0.018 
±0.017 


Teacher 
Responsiveness 

Primaty 
Secondars' 


0.67 
-0.69 


0.88 
-0.11 


1.91 
0.67 


2,00 
I.IO 


2.00 
1.88 


1.48 

0.56 


0.72 
0.93 


±0.018 
± 0.022 


Curriculum 
Usefulness 

Primary- 
Secondary 


0.67 
0.18 


0.73 
0.67 


1.98 
1.09 


2.00 
1.94 


2.00 
1.94 


1.54 
1.12 


0.76 
0.84 


±0.019 
±0.020 


Attitudes to 
Learning 

Primarv' 
Secondars- 


0.22 
1 -0.53 


0.69 
-0.08 


1.28 
0.44 


1.78 
0.94 


2.00 
1.34 


1.14 
0.42 


0.69 
0.74 


±0.017 
±0.018 
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Table 3.4 Homework and Home Help 



Variable 


10th 
%-ile 


25th 
%-iIe 


50th 
%-iIe 


75th 
%-iIe 


90th 
Vo-ile 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


£ngHsh Homework 

Primary 
Secondary' 


- 1.20 

- 1.17 


-0.72 
-0.80 


0.36 
-0.39 


1.17 
0.08 


1.72 
0.61 


0.24 
-0.34 


1.09 
0.69 


±0 027 
±0.016 


Mathematics 
Homework 

Primary 
Secondary' 


- 1.89 
-0.94 


- 1.11 

0.03 


-0.11 

0.97 


0.89 
1.03 


1.67 
1.97 


-0.20 
0.75 


1.26 
0.96 


±0.035 
± 0.024 


Home Help with 
Homework 

Primary 
Secondary 


-0.90 
- 1.49 


-0.40 
- 1.12 


0.19 
-0.46 


1.06 
0.15 


1.38 
0.75 


0.28 
-0.41 


0.92 
0.83 


±0.027 
±0.028 j 



Table 3.5 Parent Participation and Perceptions of the School 



Variable 


loth 
%-ile 


25th 
%-ile 


50th 
%-ile 


75th 
%-ile 


90th 
%-iIe 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


Assisting with 
Class Activities 

Primary 
Secondary 


-2.00 
-2.00 


- 1.75 
-2.00 


- 1.11 
-2.00 


-0.28 
- 1.71 


0.42 
- 1.17 


-0.94 
- 1.74 


0.93 
0.51 


± 0.025 
±0.015 


Attendance at 
Parent/Teacher 
Interviews, etc. 

Primar>' 
Secondary 


1.06 
-0.50 


1.46 
0.39 


1.73 
1.21 


1.87 
1.70 


2.00 
2.00 


1.57 
0.93 


0.57 
0.99 


±0.015 
±0.032 


Involvement 
in Decision- 
Making 

Primary' 
Secondary 


-2.00 
-2.00 


- 1.90 

- 1.97 


- 1.71 

- L91 


- 1.02 

- 1.67 


0.10 
-0.79 


- 1.29 

- 1.60 


0.93 
0.78 


±0.025 
±0.023 


Parent Perceptions 
of the School 

Primary' 
Secondary 


0.00 
0.00 


0.41 
0.33 


1.00 
0.84 


1.41 
1.33 


2.00 
1.84 


0.91 
0.78 


0.71 
0.73 


±0.019 
±0.021 
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Table 3.6 Teacher Affect 



Variable 


10th 
%-iIe 


25th 
%-iIe 


50th 
%-iIe 


75th 
%-ile 


90th 
%-iIe 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


Energy 


- U.oo 


U. 1 3 


U. / / 




1 .47 




U.OO 


± U.UoU 


Warmth 


0.23 


0,73 


1.16 


1,52 


1,88 


1.07 


0,64 


± 0.047 



Table 3,7 Teachers* Perceptions of their Work Environment 



Variable 


10th 
%-ile 


25th 
%-iIe 


50th 
%-iIe 


75th 
%-iIe 


90th 
%-iIe 


Mean 


S.D. 


95% 
Confidence 
Intervals 
around 
Mean 


Morale 


-0,70 


0,00 


0.62 


1,13 


1.55 


0,49 


0,87 


±0.063 


Student 
Orientation 


-0,02 


0.45 


1.00 


1.41 


2,00 


0,90 


0.74 


± 0,052 


Feedback 


- 1.55 


-0,86 


-0,21 


0.36 


0,92 


-0,26 


0,91 


±0,065 


Work Demands 


-0,64 


-0,11 


0,45 


l.IO 


1,75 


0.46 


0.88 


±0.062 


Discipline Policy 


-0,53 


0,05 


0,80 


1,25 


1,84 


0,67 


0.87 


±0,061 


Leadership 
Support 


-0,72 


O.OI 


0,71 


1,30 


1,79 


0.60 


0.95 


±0.067 


Peer Support 


-0.23 


0,35 


0.97 


1,35 


1,87 


0,83 


0.82 


± 0,058 


Role Clarity 


-0,24 


0,25 


0,78 


1,25 


1,59 


0,72 


0.71 


± 0,050 


Curriculum 
Coordination 


-0.79 


-0.08 


0.54 


1.00 


1,42 


0,40 


0,85 


±0.060 


Professional 
Development 


•«0,82 


-0,14 


0.45 


1,03 


1,50 


0,38 


0.90 


±0.064 


Goal Congruence 


-0.51 


0.08 


0,62 


1.07 


1,57 


0.56 


0,78 


± 0.055 


Decision-Making 


-0.52 


0.07 


0,61 


1,22 


1,73 


0.60 


0,84 


±0,060 



52 

4« 




Appendix 4 



RELATIONSHIPS AMONG VARIABLES 



Table 4J Relationships Among Primary Student Variables (English): 
Standardised Direct, Indirect and Total Effects 



Effects 


lyirCLl ILrllCv.l3 


InHirprt FTfprt^ 


Total Effects 1 


of Socio-Educaliona! Level 
on 

Parent Panic. (Interviews) 








0.247 


- 


0.247 


English Homework 


- 


0.050 


0.050 


Attitude to Learning 


-.091 


0.074 


-.017 


Perceived Curric. Usefulness 


0.077 


- 


0.077 


Attentivencss 


0.192 


0.006 


0.199 


English Achievement 




v. lOO 




of Non-English-Soeaking 








Background 
on 

Socio-Educational Level 








-.148 


- 


-.148 


Parent Panic. (Interviews) 


-.223 


-.037 


-.260 


English Homework 


- 


-.044 


-.044 


Attitude to Learning 


0.060 


-.013 


0.046 


Perceived Curric. Usefulness 




- 01 1 


-.029 


English Achievement 


- 


-.068 


-.068 


of Gender (Female) 
on 

English Homework 








0.091 


0.0 11 


0.102 


Attitude to Learning 


0.104 


0.113 


0.217 


Perceived Curric. Usefulness 


0.092 


- 


0.092 


Attentivencss 


0.266 


-.008 


0.274 


English Achievement 


- 


0.162 


0.162 


of Parent Participation 








(Interviews) 

on 

English Homework 














Attitude to Learning 




0.071 


0.071 


English Achievement 


0.195 


- 


0.195 


of Perceived Curriculum 








Usefulness 








on 








English Homework 


0.121 




0.121 


Attitude to Learning 


0.189 


0.068 


0.257 


Attentivencss 


0.082 




0.082 


English Achievement 




0.048 


0.048 


of English Homework 
on 

Attitude to Learning 








0.434 




0.434 


of Attentivencss 
on 

Attitude to Learning 








0.189 




0.189 


English Achievement 


0.593 




0.593 
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Table 4.2 Relationships Among Primary Student Variables (Mathematics): 
Standardised Direct, Indirect and Total Effects 



Effects 


Direct Effects 


Indirect Effects 


Total Effects 


of Socio-Educalional Level 
on 

Parent Partic. (Interviews) 








0.260 


-.003 


0.257 


Mathematics Homework 




0.011 


0.011 


Attitude to Learning 


- 1 9^ 

1 Z J 


U.U74 




Perceived Curric. Usefulness 


0.083 




0.083 


Attentiveness 


0.176 


0.022 


0.199 


Mathematics Achievement 


- 


0.039 


0.039 


of Non-EnRlish-Speaking 








Background 
on 

Socio-Educational Level 








-.145 


- 


-.145 


Parent Partic. (Inten'iews) 


-.226 


-.031 


-.257 


Mathematics Homework 








Attitude to Learning 


0.076 




.076 


Perceived Curric. Usefulness 


- 


-.012 


-.012 


Attenti\'eness 


- 


-.042 


-.042 


Mathematics Achievement 


- 


-.008 


-.008 


of Gender (Female) 
on 

Mathematics Homework 










0.015 


0.015 


/\LLiLuuc LU JLCdrnuig 


U.// 1 


u. 1/y 


U.J5U 


Perceived Curric. Usefulness 


0.112 




0.112 


Attentiveness 


0.226 


0.051 


0.277 


Mathematics Achievement 


- 


0.054 


0.054 


of Parent Particioation 








(Interviews) 








on 

Attitude to Learning 


- 


0.019 


0.019 


Attentiveness 


0.062 


- 


0.062 


Mathematics Achievement 


- 


0.012 


0.012 


of Attitude to Learning 
on 

Parent Partic. (Interviews) 








0.087 


- 


0.087 


Attentiveness 


- 


0.005 


0.005 


Mathematics Achievement 


- 


0.001 


0.001 


of Perceived Curriculum 








Usefulness 








on 








Parent Partic. (Interviews) 




0.036 


0.036 


Mathematics Homework 


0.137 


- 


0.137 


Attitude to Learning 


0.388 


0.025 


0.413 


Attentiveness 


0.077 


0.002 


0.079 


Mathematics Achievement 




0.016 


0.016 


of Attenli\'eness 








on 








Parent Partic. (Interviews) 




0.027 


0.027 


Attitude to Learning 


0.310 


0.001 


0.311 


Mathematics Achievement 


0.196 




0.196 
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Table 4*3 Relationships Among Secondary Student Variables (English): 
Standardised Direct, Indirect and Total Effects 



Effects 


Direct Effects 


Indirect Effects 


Total Ltiects 


of Socio-Educalional Level 








on 

Parent Panic. (Interviews) 


0.122 




0.122 


English Homework 


0.086 


A A 1 C 


U. lUl 


Altitude to Learning 


0.149 


0.045 


0.194 


Social Acceptance 


0.106 


0.004 


0.110 


Attentiveness 


- 


0.113 


0.113 


English Achievement 


0.380 


0.091 


0.471 


of Non-Enelish-Soeakine 








Background 
on 

Socio-Educational Level 








-.128 




-.128 


English Homework 


-.302 


-.027 


-.319 


Attitude to Learning 


-.123 


-.096 


-.218 


Social Acceptance 


-.105 


-.012 


-.117 


Attentiveness 


- 


-.127 


-. 127 


English Achievement 


- 


-.151 


1 C 1 

-.151 


of Gender HFemale) 
on 

English Homework 








0.237 


0.017 


U.253 


Attitude to Learning 


0.416 


0.066 


0.482 


Social Acceptance 


0.121 


- 


0.121 


Attentiveness 


- 


0.280 


0.280 


English Achievement 


- 


0.227 


0.227 


of Rural 








on 








Socio-Educational Level 


-.128 


- 


-.128 


English Homework 


-,302 


-.027 


-.319 


Attitude to Learning 


-,123 


-.096 


-.218 


Social Acceptance 


-.105 


-.012 


-.117 


Attentiveness 


** 


-.127 


-,127 


English Achievement 




-,151 


-,151 


of Parent Participation 








(Interviews) 

on 

Attitude to Learning 








0.068 


0.006 


0.074 


of Homework 

on 

Attitude to Learning 








0,182 




0.182 


Attentiveness 




0,106 


0.106 


English Achievement 




0.086 


0.086 


of Attitude to Learning 
on 

Attentiveness 








0.581 


- 


0.581 


English Achievement 


- 


0.470 


0.470 


of Social Acceptance 
on 

English Homework 














Attitude to Learning 


0.137 




0.192 


Attcnti\'cncss 


0.167 


0,025 


0,137 


English Achievement 




0.112 


0.112 






0.091 


0.091 
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of AUentiveness 








on 








English Achievement 


0.809 




0.809 



Table 4*4 Relationships Among Secondary Student Variables (Mathematics): 
Standardised Direct, Indirect and Total Effects 



Effects 


Direct Effects 


Indirect Effects 


Total Effects 


of Socio-Educalional Level 








on 








Parent Partic. (Interviews) 


0.104 


- 


0.104 


Mathematics Homework 


0.091 


0.016 


0.108 


Attitude to Learning 


0.119 


0.068 


0.186 


Social Acceptance 


0.120 


0.002 


0.122 


Attentiveness 




0.068 


0.068 


Mathematics Achievement 


- 


0.018 


0.018 


of Non-English-Speakin^ 








Background 
on 

Socio-Educational Level 








-.216 


- 


-.216 


Mathematics Homework 


0.207 


-.023 


0.184 


Attitude to Learning 


0.156 


-.005 


0.151 


Attentiveness 


- 


0.055 


0.055 


Mathematics Achievement 


- 


0.015 


0.015 


of Gender (Female) 
on 

Mathematics Homework 








0.131 


- 


0.131 


Attitude to Learning 


0.445 


0.019 


0,464 


Attentiveness 




0.170 


0.170 


Mathematics Achievement 


- 


0.045 


0.045 


of Rural 








on 








Socio-Educational Level 


-.157 


- 


-.157 


English Homework 


-.242 


-.017 


-.259 


Attitude to Learning 


-.140 


-.065 


-.205 


Social Acceptance 




-.019 


-.019 


Attentiveness 


_ 


-.075 


-.075 


English Achievement 


- 


-.020 


-.020 


of Parent Participation 








\ llUCrVlLWS) 








on 

Attitude to Learning 


0.045 


0.009 


0.054 


of Homework 

on 

Attitude to Learning 








0.148 




0.148 


Attentiveness 




0.054 


0.054 


Mathematics Achievement 




0.015 


0.015 


of Attitude to Learning 








on 

Attentiveness 


0.367 




0.367 


Mathematics Achievement 


■ 


0.098 


0.098 
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of Social Acceptance 








on 








Mathematics Homework 








Attitude to Learning 


0,132 




-.132 


Attentiveness 


0,387 


0.020 


0.407 


Mathematics Achievement 






0.149 








0.040 


of Attentiveness 








on 








English Achievement 


0.267 




0.267 



Table 4.5 Relationships Among Teacher Variables: 
Direct, Indirect and Total Effects 



Hit 1 Cvid 




lIlUircLl IiiilCLl5 




of Leadership Support 
on 

Peer Support 








0,924 * 


0.048 


0.972 


Role Clarity' 


0,786 


0,185 


0.970 


Professional Development 


0.992 




0.992 


Goal Congruence 


L051 




1.051 


Decision-Making 


1.010 


0.018 


1.027 


Student Orientation 




0.843 


0.843 


Curriculum Coordination 




0.982 


0.982 


E nerg)'/Enthusi asm 




0.478 


0.478 


Warmth 




0.359 


0.359 


Morale 




0.990 


0.990 


Feedback 




0.913 


0,913 


Work Demands 




-.268 


-.268 


Disciphne Policy 




0,819 


0.819 


of Goal Congruence 
on 

Morale 








0.607 




0,607 


Discipline Polic>' 


0.361 


0.286 


0.647 


Student Orientation 


0.758 




0.758 


Curriculum Co-ordination 


0.638 




0.638 


of Professional De\'cloDmcnt 
on 

Energy- 
Feedback 
Work Demands 


0.443 
0.921 
-.270 


0.039 


0.482 
0.921 
-.270 
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Table 4.6 Predictors of Teacher Morale: 



Predictors of Morale 
(Fixed Effects) 


Standardised Regression 
Coefficients 


Standard Errors 


Goal Congruence 


0.423 


0.030 


Leadership Support 


0.225 


0.031 


Peer Support 


0.234 


0.029 


Average School 
Leadership Support 


0.111 


0.032 


Proportion of variance in 
Morale explained by: 

fixed effects: 
other teacher effects: 
school-level effects: 


72.7% 
23.1% 
4.2% 
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TECHNICAL NOTES 



Appendix 5 



Calculation of composite variables and estimating their reliabilities using one-factor 
congeneric models 

A composite variable is a single index of several constituent indicator variables, each of wliich 
has been weighted for its relative contribution to the composite. Composite variables in this study 
were calculated from one-factor congenric measurement models, using LISREL 7 (Joreskog & 
Sorbom, 1989). Under a listwise method of deleting missing data, each one-factor model was 
analysed using a weighted least squares method of parameter estimation, based cm a 
tetrachoric/polychoric/polyserial/product-moment intercorrelation matrix and an asymptotic 
covariance matrix of these correlations computed from PRELIS (Joreskog & Sorbom, 1988). 
The measurement model for calculating a composite of the x variables may be written as 

x = A^^,+6, (1) 
and the variance-covariance matrix of x is 

s = ^^<^^; + ©5 (2) 

From equation (1) x is a (^7 x 1) vector of measurements on observed indicators, is a x 1) 
vector of fixed coefficients or loadings on an underlying latent (composite) variable (^j) and 6 is a 
(w X 1) vector of unique (random) factors specifying the measurement errors in the x indicator 
variables. From equation (2) E is the estimated variance-covariance matrix of the factor loadings 
(X^) for the vector of congeneric measures (x), O is the correlation among the factors (for a one- 
factor model, O = 1) and 05 a vector of unique variances. 

The reliability of an unweighted composite is given by 



= ^ . and (3) 

z 



to maximise the reliability of the composite. 



rc= -> (4) 



where is a vector of factor score regression weights (Wj = ^xi^^sv ^2 ^x2^^S2^ 
maximise the reliability of the composite. The reliability coefficients for all composite variables 
employed in the study, using equation (4), are provided in Tables 2. 1 and 2.2 of Appendix 2. 



Calibration of indicators for the English and Mathematics profiles using item 
response methodology 

Detailed accounts of the procedures emploN'cd to develop the student achievement profiles used in 
tills study have been provided by Griffin (1990), Griffin & Nix (1991) and Rowe (1992). For 
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illustrative purposes, following is a brief outline of those procedures as they relate to the 
calibration of the reading profiles. 

Lists containing behavioural indicators of reading proficiency were provided to teachers in 22 
schools. At the instigation of these teachers, a rating scale was introduced to indicate the level of 
confidence they had that any one indicator was exhibited by a student. Teachers correctly argued 
that the development of these indicators was more complex than a simple "present" or "absent" 
rating of a student's behaviour. By using a "partial credit" rating scale of 0, 1 and 2 to describe 
increasing evidence of an indicator being exhibited, teachers allowed for this possibility. Hence, 
for each indicator a zero (0) rating was used if the teacher had not observed a student exhibiting a 
performance indicator; a rating of one (1) was used to indicate that the behaviour was still 
developing and had yet to be consistently displayed by the student; and a two (2) was used if the 
teacher was confident that the indicator was an established part of the student's repertoire of 
reading behaviours. For practical purposes, the rating observation categories were labelled as 
"No" (0), "Maybe" (1) and "Yes" (2). 

Data from more than 1200 students for the indicators were analysed using item response 
methodology' as it applies to partial credit ratings (Adams & Khoo, 1992; Andrich, 1978; 
Masters, 1982). From Masters (1982), the probability (7C^;„) of student n being rated x on the m^- 
step (0, 1 or 2) for indicator / may be expressed as a function of the student's position on the 
developmental attribute of reading proficiency and the difficulties (6,y) of the /w,-steps as follows: 

^xn, = —, ^^1 x = 0,\,2 (/72,-steps), (5) 

where k refers to the number of individual step difficulties (two in this case). This enabled the 
indicators to be calibrated on a linear developmental scale (or growth continuum) expressed in 
logKs, ranging from approximately -5 (easiest and lowest level of attribute development) to +5 
(most difficult and highest level of attribute development). 

The full list of indicators obtained from the analyses was examined in terms of their respective 
logit values on the developmental scale and for patterns that might be useful in summarising them 
into groups. It was decided to group the indicators into nine bands, labelled "A" to "1", with each 
group of indicators having a logit value range of approximately 1,5 logits. Band A was 
deliberately set at the eariicst developmental level to avoid the association of "value" with 
development. Due to the empirical calibration of their constituent indicators using equation (5), 
the bands were designed to be cumulative. This is an important feature of the profiles which 
indicates that for a student whose mother tongue is English and is developing proficiencies at 
band E, for example, the student would also be likely to have established the behaviours described 
at lower band levels, but unlikely to have established proficiency at higher levels. For non- 
English speaking background students with wcll-dcvclopcd reading skills in their mother tongue, 
trials of the reading profile bands indicated a pattern that showed "beginning" development across 
several band levels (e.g., Rowe, 1991a). 



Structural equation models for examining the relationships among student and 
teacher factors. 

Separate models were fitted to the data for students and teachers using the LISREL method for 
Sub model 3b (Jorcskog & Sorbom, 1989:189-190), This general model contains only y 
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(observed) and r| (latent or composite) variables. In fitting such models it was h>pothesised that 
all variables essentially form part of an interdependent, endogenous system. 



The structural relationships among the latent (composite) variables (r|) are given by 

t1 = Bti+^ (6) 

and the covariance matrix of ^cy variables is 

Z = Av<I - B)-l^(I - BYAy' + 08 (7) 

This model has only four parameter matrices, namely. Ay , ^ and 0s, where Ay is the 
matrix of factor loadings for the y variables (absorbed in the one-factor composite variables 
denoted by r|), B is the matrix of directional relationships among the latent endogenous constructs 
(il), W is the matrix of variances and covariances among the residuals of the r| variables, and 0s 
is the matrix of variances and covariances among the measurement errors s associated 
with the y variables. The major interest in the models fitted for this study concerned the 
magnitude estimation of the direct, indirect and total effects for the parameters of B, provided in 
Tables 4.1 to 4.5 of Appendix 4. 

For each of the structural equation models fitted, indices of the estimates for the fixed parameters 
(X^) of the composite variables were calculated from = yjr^ , (from equation 4) and the fixed 
parameter estimates for the composites' unique (error) variances (0^) were derived from 
®c ~ (1 ■ ''c)- specific details of these procedures, see Alwin & Jackson (1980), Brown 
(1989), Jorcskog (1971), Munck (1979), Werts et ai (1978). 

The fit statistics for each of the models shown diagrammatically in Figures 10 to 12 are shown in 
Table 5.1 below. 

Table 5.1 Goodness-of-fit statistics for structural equation models estimating 
the relationships among student and teacher factors 



Figure 


Model 


N 


X2 


d.f. 


P 


GFI 


AGFI 


RMR 


10 


Relationships among 
factors for primary 
students (English) 


3913 


5.82 


16 


0.971 


1,000 


0.999 


0.017 


10 


Relationships among 
factors for primar>' 
students (Maths) 


3859 


7.84 


17 


0.970 


1.000 


0.999 


0.046 


11 


Relationships among 
factors for secondary' 
students (English) 


2918 


8.75 


14 


0.841 


1.000 


0.998 


0,058 


11 


Relationships among 
factors for secondary- 
students (Maths) 


2582 


22.43 


17 


0.160 


0.999 


0.996 


0.041 


12 


Relationships among 
teacher factors 


678 


66.61 


58 


0.205 


0.995 


0.992 


0.043 



OKI - Goodnes-of-fil index; AGFI - Adjusted goodiiess-o f-fu index; RMR - Root mean square residual 
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Multilevel models for partitioning the variance in response variables ofinterestj due 
to student- teacher- and school-level effects 

The method used for estimating the proportion of variance in Jie response variables of interest 
(namely, student achievement outcomes) due to the clustering effects of students within 
classes/teachers within schools, was to fit simple two- and three-level models. 

For each of the four response variables of interest, namely Primary English, Primary 
Mathematics, Secondary English and Secondary Mathematics, four models were fitted to the 
data: 

(a) Two-level variance-components model (hereinafter referred to as "null model"), to estimate 
the variance due to the group effect of students (level 1) within schools (level 2), without any 
explanatory variables; 

(b) Two-level model, as for (a), and student intake characteristics as fixed, level I explanatory 
variables; 

(c) Three-level null model to estimate the variance due to the group effects of students (level 1) 
within clascsses/teachers (level 2), within schools (level 3) without any explanator>' variables; 

(d) Three-level model, as for (c), and student inUike characteristics as fixed, level 1 explanatory 
variables. 

The equations for the null models (a) and (c) may be illustrated with reference to the three-level 
model (c). Following Prosser, Rasbash & Goldstein (1991), this model was written in three parts. 
First, for the /th student in class/teacher / within school k, scores on English and Mathematics 
achievement (Yijk), were modelled as 

^ijk = Poy^Xo + ejjk. (8) 

In this model the slope is constant (0 - zero) bat the intercept is random, var>ang across classes/ 
teachers and schools. The Xq term in equation (8) is a column vector of unities representing the 
constant slope for classes/teachers and schools, and ejjk is a random residual term representing 
the contribution to the response variable Yjjk of the /th student in theyth class/teacher within the 
kth school. 

Second, the intercept for class/teacher Jk can be expressed as a linear function of the average 
intercept for school k (tiqq^.) and a class/tcachcr-lcvel random term {liQjk)' 

^Qjk = T^QQk (9) 

Third, the average intercept for school k can be modelled in tenns of an overall school average 
intercept (Yooo)' ^ school-level random tcnn (voojt)* 

^ooA: = Yooo + ^oo^. 

By combining equations (8), (9) and (10), a single equation version of the model can be written as 
follows: 

Y;yi = Yooo + (^ooit ^- uojk + eijk\ (11) 
where y^^o the fixed part of the model and the three random terms are bracketed. 
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From equation (11), given the fixed part of the model (yooo - the grand mean of Y/yjt), the random 
parameters that were estimated for this model were the variances of the residual terms in brackets, 
namely: 

• the between-school variance estimate of the intercept (i.e., 0^3^ for vook)'. 

• the between-<:lasses/teachers variance of the intercept (i.e., o^^) ^Qjk)y and; 

• the between-students, within classes/teachers and schools variance of the residual term 
(i.e., G^^^ (or ejjk). 

The total variance due to random effects (a^j.) = 0^3^ + C7^2) ^i)) "^Y then be partititoned 
into that due to school, class/teacher and student effects as follows: 

Proportion of variance due to: 

school effects: ^o)^^t) 
class/teacher effects: 0^2)^ 
student effects : ^o)^^(T) 

To adjust for student intake characteristics, in models (b) and (d), four explanatory variables 
were introduced as level 1 (student-level) fixed parameters. This may be illustrated with 
reference to the three-level model, in which the variables of, socio-educational level, gender, non- 
English-speaking background and rurality were fitted as explanatory variables and modelled as: 

Y/y/r = yoooXo+ Jioo^Ujk + 7200X2//'/: + y^oo^^ijk + Y400X4///: + 

(voo/: + "o/Jt + ^/y/:), (12) 

where the variables X^ijk, y^iijk^ ^vjk and X^jjk are the student-level measures of socio- 
educational level (SEL), gender (female) non-English-speaking background (NESB) and rurality 
respectively, and yioo to 7400 are their related parameters to be estimated, yooo is grand mean of 
the response variable of interest and Xq is the intercept variable that takes the value 1 for all 
students. 

Under an iterative generalised least squares method of estimation (see Goldstein, 1986), the 
models described by equations (1 1) and (12) were fitted to tlie data using MLS (Prosser, Rasbash 
& Goldstein, 1991). The relevant results are summarised in Tables 4.7 and 4.8 (pp. 24-25), and 
shovN-n graphically in Figure 14 (p. 25). Parameter estimates and their standard errors are given 
in Tables 5.2 and 5.3 in Appendix 5. 

To illustrate variation in student achievement due to school effects (based on the data presented in 
Table 5.2), Figure 17 shows the two-level (students witliin schools), intake-adjusted, within-group 
regression lines for English achievement in each of the 57 primary schools. Each school is 
represented by a line segment whose end points indicate the school's minimum and maximum 
score. [To facilitate interpretation, individual students' English achievement scores (horizontal 
axis) and school average English achievement scores (vertical axis) have been standardised (Z- 
scores)]. The vertical distance from the horizontal axis to a given line segment is equal to the 
mean achievement for that school. 

Similarly, to illustrate variation in student achievement due to class/leacher effects (from the data 
presented in Table 5.3), Figure 18 shows the intake-adjusted, within-group regression lines for 
English achievement in each of the 322 primar>' school class/tcachcr groups, with class average 
English achievement (standardised) shown on the vertical axis. 
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Table S.2 Standardized Parameter Estimates for Two-Level Models 
of School and Student Effects 



Primary 


Primaiy 


Secondary 


Secondary 


English 


Maths 


English 


Maths 




4205 students 


3338 students 




57 schools 


57 schools 


30 schools 


28 schools 


Parameter Est. SE 


Est. 


SE 


Est. 


SE 


- 

Est. SE 


Null Model (Variance Components): 












Random 












J2 / « i\ n AAA* /I /T5 T 

0^2) (school) 0.C99* 0.022 


0.123* 


0.026 


0.097* 


0.029 


U.lUo U.UJj 


o^,) (student) 0.900* 0.020 


0.886* 


0.019 


0.914* 


0.022 


0.923* 0.024 


CTj, (total) 0.999 


1.009 




1.012 




1.031 


Intake-Adjusted Model: 












Fixed 












V f^FT 0 182* 


0.171* 


0.016 


0.175* 


0. 022 


0.200* 0.020 


(2-;nder. female) 0.141* 00/5 


0.031* 


0.015 


0.209* 


0.018 


0.031 0.019 


/jo (NESB) -.047* 00/7 


0.001 


0.017 


-.026 


0.018 


0.066* 0.020 


(rural) 0.029 0.035 


0.112 


0.038 


.017 


0.022 


0.056* 0.024 


Random 












o^,, (school) 0.086* 00/9 


0.111* 


0.02^ 


0.066* 


0.020 


0.084* 0.026 


0^,) (student) 0.852* 00/9 


0.861* 


0.019 


0.856* 


0.021 


0.892* 0.023 


rr^ ftr^tr^n 0 938 


0.972 




0.922 




0.975 


o (iixeu eitects) * "'^^ * 


0.037 




0.089 






Proportion of variance in unadjusted scores due to: 












Intake efiects .061/.999=.061 


.037/1 .009=.037 


.089/1.012=088 


.055/1.031=054 


School effects .086/.999=.086 


.111/I.009=.I10 


.066/1. 012=.066 


.084/1. 031 =.081 


Other student effects .852/.999=.853 


.861/1.009=853 


.856/1.012=846 


.892/1.031=865 


Proportion of variance in intake-adjusted scores due to: 










School effects .086/.938=.092 


.111/.972 


= 114 


.066/.922 


=.072 


.084/.975=.086 


Other student effects .852/ 938= 908 


.861/.972 


= 886 


.856/.922 


= 928 


.892/.975=.914 


^ Estimated as Uie difference between O^^^ (total) for the null and intake-adjusted models 



• Statistically significant beyond the p < 0.05 level, by univariate two- tailed test. 

84 
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Table 5.3 Standardized Parameter Estimates for Three-Level Models 
of School, Class/Teacher and Student Effects 





Primary 


Primary 


Secondary 


Secondary 




English 


Maths 


English 


Maths 




4205 students 


4205 students 


3338 students 


2934 students 




322 classes 


322 classes 


216 classes 


182 classes 




57 schools 


57 schools 


30 schools 


28 schools 


Parameter 


Est 


SE 


Est 


SE 


Est 


SE 


Est SE . 


Null Model (Variance Components): 












Random 
















0(3) (scnool) 






0.028 


0,022 


0.047 


0,059 


0.000 0.000 


a^2) (class/teacher) 


0.309* 


0.032 


0.447* 


0.043 


0.279* 


0,035 


0.391* 0.047 


o^i) (student) 


U.o44^ 


U.UIJ 


0.537* 


0.012 


0.701* 


0,018 


0.615* 0.016 


o^yj (total) 


A /\AA 

0.990 




1.012 




1.027 




1.006 


Intake-Adjusted Model: 
















Fixed 
















/^oo(SEL) 


0.181* 


0.014 


0.165* 


0,013 


0.170* 


0.017 


0.183* 0017 


/200 (gender: female) 


0.136* 


0.012 


0.024* 


0.012 


0.196* 


0,017 


0.026* 0.016 


^300 (NESB) 


-.057* 


0.014 


-.010 


0,014 


-.031 


0.017 


0.048* 0.017 


^400 (rura^) 


-.017 


0.014 


0,039 


0,038 


-.019 


0.02 J 


0.057* 0.021 


Random 
















a^3j (school) 


0.021 


0.017 


0.014 


0,019 


0.032 


0,021 


0.000 0.000 


o^2) (class/teacher) 


0.309* 


0.031 


0.445* 


0.042 


0.267* 


0,033 


0.373* 0.045 


o^ij (student) 


0.597* 


0.014 


0.516* 


0,012 


0.649* 


0,016 


0,590* 0.016 


o^D Ootal) 


0.927 




0,975 




0.948 




0.962 


(T (fixed effects)^ 


0.063 




0.037 




0.079 




0.043 



Proportion of variance in imadjustcd scores due to: 

Intake effects .063/.990=.063 .037/1 0]2=.036 .079/1. 027=.077 .04 3/ 1.006=. 04 3 

School effects .021/.990= 021 .014/1.012=014 .032/1.027=031 .000/1.006=000 

Class/teacher eflects .309/.990=.3 12 .445/1 .012=.440 .267/1 .027=.260 .373/1 .006=.370 

Other student effects .597/.990=.585 .516/1. 012= 510 .649/1.027= 632 .590/1. 006= 586 

Proportion of variance in intake-adjusted scores due to: 

School effects .021/.927=.023 .014/.975=.015 .032/.948=.034 .000/.962=.000 

Class/teacher effects .309/.927=.333 .445/.975=.456 .267/.948=.281 .373/.962=.387 

Other student effects .597/.927=.644 .516/.975=.529 .649/.948=.685 .590/.962= 613 



^ Estimated as the difference between O^^j (loml) for tlie null and intake^adjustcd models 
• Statistically significant lx:yond the p < 0.05 level, by univariate two-iailcd lest. 



S5 

59 



PRIMARV KHCUSH 



Figure 17. Variation in English achievement for 57 primary schools 




English^ 
PRIHARV £NCLISH 

Figure 18. Variation in English achievement for 322 class/teacher groups 



BG 

^ BEST COPY MMUBLE 



Two-level models were also fitted to the teacher data, both for the purposes of predicting teacher 
morale in terms of teacher and school-level explanatory variables (Figure 13, p. 23 and Table 
4.6, p. 52) and for estimating the proportion of variance in the dimensions of teachers' affect and 
perceptions of their work environment due to both teacher- and school-level effects. Table 5.4 
summarises the results of fitting variance components models for each of the affect and work 
environment dimensions, to estimate the proportion of variance in each dimension due to school 
effects. 



Table 5.4 Per cent of Variance in Teacher Affect and Work Environment 
Dimensions Due to Between-School Differences 



Dimension 


Primary 


Secondary 


322 teachers* 


356 teachers* 




57 schools 


27 schools 


Energy/Enthusiasm 


9.1 


6.9 


Warmth (towards students) 


0.0 


3.6 


Morale 


33.2 


18.5 


Feedback 


19.4 


10.5 


Work Demands 


24.2 


9.5 


Leadership Support 


28.6 


14.2 


Discipline Policy 


28.8 


29.9 


Peer Support 


20.6 


10.2 


Role Clarity 


11.4 


7.1 


Professional Development 


23.8 


17.1 


Goal Congruence 


30.7 


19.4 


Decision Making 


20.9 


12.1 


Student Orientation 


24.9 


14.5 


Curriculum Coordination 


27.0 


8.5 



* Teacher questionnaire returns with complete data 



Table 5.5 presents the detailed results of the two-level model fitted to predict teacher morale in 
terms of the teacher and school-level explanatory' variables illustrated in Figure 13 (p. 23) and 
summarised in Table 4.6 (p. 52). 
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Table 5.5 Variation in Teacher Morale due to School- and Teacher-Level Effects 
Showing Unstandardised and Standardised Parameter Estimates and 
Standard Errors (678 teachers in 84 schools) 



Explanatory Variables/ 
Parameters 



Unstandardised 
Est. SE 



Standardised 
Est. SE 



Null Model: (Random Parameters) 

a^2) (school) 
(7^,) (teacher) 
(total) 



0.34* 0.07 0.38* 0.08 
0.63* 0.04 0.70* 0.04 
0.97 1.08 



Proportion of variance in Morale 
due to: school efTects .352 

teacher eflecls ,648 



Explanatory Model: 

FIXED 



Yqq (Constant) 


3.70 


0.03 


0.01 


0.03 


Y,Q (Leadership support) 


0.21* 


0.03 


0.23* 


0.03 


Y20 (Peer support) 


0.25* 


0.03 


0.23* 


0.03 


Y30 (Goal congruence) 


0.45* 


0.03 


0.42* 


0.03 


Yoi (Sch. av. Leader, support) 


0.17* 


0.05 


0.11* 


0.03 


RANDOM 










cr^-,) (school) 


0.04* 


0.01 


0.05* 


0.01 


CJ^ij (teacher) 


0.22* 


0.01 


0.25* 


0.01 


ctIt) (total) 


0.26 




0.30 




cT (fixed effects)' 


0.70 




0.78 




Proportion of variance in Morale- 










duo to: fixed cfleets 


.701/966= 


--.126 


.784/1 .079 


= 726 


school etTects 


.042/.966= 


=.043 


.046/1 079 


= 04.3 


otlwr teacher eflecls 


.22.V.966= 


=.231 


.249/1.079 


=.231 



I:slimalcd as Uic di (Terence between O^j^^ (total) for the null and explanatorv nuxlels 
* Statistically signillcant beyond \l\cp< 0 05 level, by univariate two-tailed test 



Notes on multi-level and structural equation modelling 

One of the advantages of fitting structural equation models (SEM) to data of the present kind is 
that they can account for measurement error in both the observed and latent variables, as well 
as provide a means of estimating the magnitude of direct, indirect and interdependent effects 
among those latent variables. Despite these advantages, however, SEM approaches to analysis 
assume that the sample variables measured, regardless of level, are mdependently distributed in 
a multivariate population. That is, a SEM model assumes single-level data, and there is no 
simple adjustment of the structural modelling framework that can be made in order to deal with 
non-independent observations (Cuttance & Ecob, 1987). 

On the other hand, despite the utility of fitting multi-level models to hierarchically structured 
data, t\pical of school effectiveness research, current multi-level modelling computer program 
applications also have limitations. Drawbacks in their use at this stage are that they cannot 
cope with the estimation of interdependent effects among multiple response and explanatory 
variables (observed or latent); moreover, like the simple general linear model, they assume that 
explanatory variables are measured without error In fact, Scheerens ( 1 993 :3 1 ) notes: 

The usual LISREL-t>pe and multi-level models that are applied to the analysis of school 
effectiveness research data are considered to be closed models. When techniques that can 
handle the combination of these two approaches become available, we will have come a long 
way in closing the gap between conceptual and formal mathematical models of school 
effectiveness. 



Nevertheless, important theoretical work on developing techniques to analyse multi-level 
covariance structure data has already been done {e.g. Goldstein & McDonald, 1988; 
McDonald & Goldstein, 1989) and the availability of relevant computer software packages is 
imminent. 
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